{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. 观察Otto商品的特征进行PCA各维的方差，可以得到什么结论？（20分）\n",
    "2. 对Otto商品tfidf特征，进行PCA降维，给出各维方差的分布图。（30分）\n",
    "3. 采用train_test_split，从将数据集中随机抽取10000条记录（原始数据集太大，剩余数据抛弃，此部分SVM作业已经完成）。对这部分数据进行PCA降维，保留85%的能量。（20分）\n",
    "4. 对3中得到的数据（对降维后的数据），训练RBF核SVM，并对超参数（C和gamma）进行超参数调优。结果和用原始数据的情况比较（SVM部分作业结果）。（30分）\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1   观察Otto商品的特征进行PCA各维的方差，可以得到什么结论？（20分）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在去总体能量0.85后得到的方差分布如图，原先的93维数据特种降低到33维。图中前四个方差占总体能量较大。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![image.png]\n",
    "\n",
    "[image.png]:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2   对Otto商品tfidf特征，进行PCA降维，给出各维方差的分布图。（30分）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn.decomposition import PCA\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>feat_1_tfidf</th>\n",
       "      <th>feat_2_tfidf</th>\n",
       "      <th>feat_3_tfidf</th>\n",
       "      <th>feat_4_tfidf</th>\n",
       "      <th>feat_5_tfidf</th>\n",
       "      <th>feat_6_tfidf</th>\n",
       "      <th>feat_7_tfidf</th>\n",
       "      <th>feat_8_tfidf</th>\n",
       "      <th>feat_9_tfidf</th>\n",
       "      <th>...</th>\n",
       "      <th>feat_85_tfidf</th>\n",
       "      <th>feat_86_tfidf</th>\n",
       "      <th>feat_87_tfidf</th>\n",
       "      <th>feat_88_tfidf</th>\n",
       "      <th>feat_89_tfidf</th>\n",
       "      <th>feat_90_tfidf</th>\n",
       "      <th>feat_91_tfidf</th>\n",
       "      <th>feat_92_tfidf</th>\n",
       "      <th>feat_93_tfidf</th>\n",
       "      <th>target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0.081393</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.075886</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Class_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.231403</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Class_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.199730</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Class_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>0.011987</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.011668</td>\n",
       "      <td>0.105971</td>\n",
       "      <td>0.021681</td>\n",
       "      <td>0.080435</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.008244</td>\n",
       "      <td>0.022456</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Class_1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.124622</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.145988</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Class_1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 95 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   id  feat_1_tfidf  feat_2_tfidf  feat_3_tfidf  feat_4_tfidf  feat_5_tfidf  \\\n",
       "0   1      0.081393           0.0           0.0      0.000000      0.000000   \n",
       "1   2      0.000000           0.0           0.0      0.000000      0.000000   \n",
       "2   3      0.000000           0.0           0.0      0.000000      0.000000   \n",
       "3   4      0.011987           0.0           0.0      0.011668      0.105971   \n",
       "4   5      0.000000           0.0           0.0      0.000000      0.000000   \n",
       "\n",
       "   feat_6_tfidf  feat_7_tfidf  feat_8_tfidf  feat_9_tfidf   ...     \\\n",
       "0      0.000000      0.000000      0.000000           0.0   ...      \n",
       "1      0.000000      0.000000      0.231403           0.0   ...      \n",
       "2      0.000000      0.000000      0.199730           0.0   ...      \n",
       "3      0.021681      0.080435      0.000000           0.0   ...      \n",
       "4      0.000000      0.000000      0.000000           0.0   ...      \n",
       "\n",
       "   feat_85_tfidf  feat_86_tfidf  feat_87_tfidf  feat_88_tfidf  feat_89_tfidf  \\\n",
       "0       0.075886       0.000000       0.000000            0.0            0.0   \n",
       "1       0.000000       0.000000       0.000000            0.0            0.0   \n",
       "2       0.000000       0.000000       0.000000            0.0            0.0   \n",
       "3       0.000000       0.008244       0.022456            0.0            0.0   \n",
       "4       0.124622       0.000000       0.000000            0.0            0.0   \n",
       "\n",
       "   feat_90_tfidf  feat_91_tfidf  feat_92_tfidf  feat_93_tfidf   target  \n",
       "0       0.000000            0.0            0.0            0.0  Class_1  \n",
       "1       0.000000            0.0            0.0            0.0  Class_1  \n",
       "2       0.000000            0.0            0.0            0.0  Class_1  \n",
       "3       0.000000            0.0            0.0            0.0  Class_1  \n",
       "4       0.145988            0.0            0.0            0.0  Class_1  \n",
       "\n",
       "[5 rows x 95 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train=pd.read_csv('Otto_FE_train_tfidf.csv')\n",
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_train=train['target']\n",
    "X_train=train.drop([\"id\",\"target\"],axis=1)\n",
    "\n",
    "train_id=train['id']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "#PCA降维\n",
    "pca=PCA(n_components=0.85)\n",
    "pca.fit(X_train)\n",
    "\n",
    "X_train_pca=pca.transform(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<BarContainer object of 48 artists>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD8CAYAAACb4nSYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAEA9JREFUeJzt3H+s3fVdx/Hny3bA3CJsUM1swXahmnUZTnfXLXHqMpQVmesSIZb5oyaYajLMjJuzGMe2OhMwZt0SMbEZaGVqIcwfN1IlBOaPLAvrZbgfHZLdIY5ryShrx8QFurK3f5wv2fFwy/3ee8/t5d7P85E0fL+f7+d7zucTcl+fz/mc7/mkqpAkteG7lrsBkqTTx9CXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNWTtcjdg1HnnnVcbN25c7mZI0opy7733PlZV6+aq97wL/Y0bNzI1NbXczZCkFSXJf/Wp5/KOJDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1pNcvcpNsAz4CrAE+WlXXjVz/CeDDwEXAjqq6bejaTuD3utMPVtX+cTT8VDbuvv1ZZQ9dd9lSvqUkrRhzzvSTrAFuAC4FtgBXJtkyUu0rwK8AfzVy70uB9wGvA7YC70vyksU3W5K0EH2Wd7YC01X1YFWdAA4A24crVNVDVfU54Nsj974ZuLOqjlXVceBOYNsY2i1JWoA+ob8eeHjofKYr62Mx90qSxqxP6GeWsur5+r3uTbIryVSSqaNHj/Z8aUnSfPUJ/Rng/KHzDcCRnq/f696q2ldVE1U1sW7dnNtBS5IWqE/oHwI2J9mU5AxgBzDZ8/XvAC5J8pLuC9xLujJJ0jKYM/Sr6iRwNYOwvh+4taoOJ9mT5K0ASV6bZAa4AvjTJIe7e48Bv89g4DgE7OnKJEnLoNdz+lV1EDg4Unbt0PEhBks3s917E3DTItooSRoTf5ErSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1Jakiv0E+yLckDSaaT7J7l+plJbumu35NkY1f+giT7k3w+yf1Jrhlv8yVJ8zFn6CdZA9wAXApsAa5MsmWk2lXA8aq6ENgLXN+VXwGcWVWvAl4D/NozA4Ik6fTrM9PfCkxX1YNVdQI4AGwfqbMd2N8d3wZcnCRAAS9KshZ4IXAC+MZYWi5Jmrc+ob8eeHjofKYrm7VOVZ0EHgfOZTAA/C/wCPAV4I+q6tjoGyTZlWQqydTRo0fn3QlJUj99Qj+zlFXPOluBp4HvBzYB70ry8mdVrNpXVRNVNbFu3boeTZIkLUSf0J8Bzh863wAcOVWdbinnbOAY8Hbgn6rqW1X1KPBJYGKxjZYkLUyf0D8EbE6yKckZwA5gcqTOJLCzO74cuLuqisGSzpsy8CLg9cB/jKfpkqT5mjP0uzX6q4E7gPuBW6vqcJI9Sd7aVbsRODfJNPBbwDOPdd4AvBj4AoPB48+q6nNj7oMkqae1fSpV1UHg4EjZtUPHTzJ4PHP0vidmK5ckLQ9/kStJDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDWk1947q8HG3bc/q+yh6y5bhpZI0vJxpi9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1pFfoJ9mW5IEk00l2z3L9zCS3dNfvSbJx6NpFST6V5HCSzyc5a3zNlyTNx5yhn2QNcANwKbAFuDLJlpFqVwHHq+pCYC9wfXfvWuBjwK9X1SuBNwLfGlvrJUnz0memvxWYrqoHq+oEcADYPlJnO7C/O74NuDhJgEuAz1XVZwGq6mtV9fR4mi5Jmq8+ob8eeHjofKYrm7VOVZ0EHgfOBX4QqCR3JPlMkvcsvsmSpIVa26NOZimrnnXWAm8AXgt8E7gryb1Vddf/uznZBewCuOCCC3o0SZK0EH1m+jPA+UPnG4Ajp6rTreOfDRzryv+lqh6rqm8CB4EfHX2DqtpXVRNVNbFu3br590KS1Euf0D8EbE6yKckZwA5gcqTOJLCzO74cuLuqCrgDuCjJd3eDwU8CXxxP0yVJ8zXn8k5VnUxyNYMAXwPcVFWHk+wBpqpqErgRuDnJNIMZ/o7u3uNJPsRg4CjgYFXdvkR9kSTNoc+aPlV1kMHSzHDZtUPHTwJXnOLejzF4bFOStMz8Ra4kNcTQl6SG9FreWc027n72VwwPXXfZMrREkpaeM31JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDWl+G4bn4hYNklYbZ/qS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNcZfNBXD3TUkrlTN9SWqIoS9JDTH0Jakhhr4kNaRX6CfZluSBJNNJds9y/cwkt3TX70myceT6BUmeSPLu8TRbkrQQc4Z+kjXADcClwBbgyiRbRqpdBRyvqguBvcD1I9f3Av+4+OZKkhajz0x/KzBdVQ9W1QngALB9pM52YH93fBtwcZIAJHkb8CBweDxNliQtVJ/QXw88PHQ+05XNWqeqTgKPA+cmeRHwO8AHFt9USdJi9Qn9zFJWPet8ANhbVU885xsku5JMJZk6evRojyZJkhaizy9yZ4Dzh843AEdOUWcmyVrgbOAY8Drg8iR/CJwDfDvJk1X1x8M3V9U+YB/AxMTE6IAiSRqTPqF/CNicZBPw38AO4O0jdSaBncCngMuBu6uqgB9/pkKS9wNPjAa+JOn0mTP0q+pkkquBO4A1wE1VdTjJHmCqqiaBG4Gbk0wzmOHvWMpGS5IWpteGa1V1EDg4Unbt0PGTwBVzvMb7F9A+SdIY+YtcSWqIoS9JDXE//TFyn31Jz3fO9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGuLTO6eJT/ZIej5wpi9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiD/OWmb+aEvS6eRMX5Ia4kz/eWq2TwDgpwBJi+NMX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kN6bXhWpJtwEeANcBHq+q6ketnAn8BvAb4GvDzVfVQkp8GrgPOAE4Av11Vd4+x/U1yO2ZJCzXnTD/JGuAG4FJgC3Blki0j1a4CjlfVhcBe4Pqu/DHgZ6vqVcBO4OZxNVySNH99lne2AtNV9WBVnQAOANtH6mwH9nfHtwEXJ0lV3VdVR7ryw8BZ3acCSdIy6BP664GHh85nurJZ61TVSeBx4NyROj8H3FdVT42+QZJdSaaSTB09erRv2yVJ89RnTT+zlNV86iR5JYMln0tme4Oq2gfsA5iYmBh9bfXkWr+kufQJ/Rng/KHzDcCRU9SZSbIWOBs4BpBkA/C3wC9X1ZcX3WItiAOCJOi3vHMI2JxkU5IzgB3A5EidSQZf1AJcDtxdVZXkHOB24Jqq+uS4Gi1JWpg5Z/pVdTLJ1cAdDB7ZvKmqDifZA0xV1SRwI3BzkmkGM/wd3e1XAxcC703y3q7skqp6dNwd0cL4CUBqS6/n9KvqIHBwpOzaoeMngStmue+DwAcX2UZJ0pj0Cn21x08A0urkNgyS1BBDX5Ia4vKO5s2lH2nlcqYvSQ0x9CWpIS7vaGxc9pGe/5zpS1JDnOlryc32CQD8FCAtB2f6ktQQZ/paVn4PIJ1ehr6el55rMHCgkBbO0Neq4WAgzc3Q16rnpwbpOwx9aRanGgwcJLTSGfrSmDggaCUw9KUl5mCg5xNDX1omz/WjNQcKLRVDX1pBFvKltAOIhhn6UqPGOYA4sKwchr6kJbWQgcJBZOkY+pJWDAeQxTP0JTWp1QHE0JekRZrPAPJc107HAOLWypLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNaRX6CfZluSBJNNJds9y/cwkt3TX70mycejaNV35A0nePL6mS5Lma87QT7IGuAG4FNgCXJlky0i1q4DjVXUhsBe4vrt3C7ADeCWwDfiT7vUkScugz0x/KzBdVQ9W1QngALB9pM52YH93fBtwcZJ05Qeq6qmq+k9guns9SdIy6BP664GHh85nurJZ61TVSeBx4Nye90qSTpNU1XNXSK4A3lxVv9qd/xKwtap+Y6jO4a7OTHf+ZQYz+j3Ap6rqY135jcDBqvr4yHvsAnZ1pz8EPDCGvp0HPDaG11mpWu5/y30H+99q/3+gqtbNVanPhmszwPlD5xuAI6eoM5NkLXA2cKznvVTVPmBfj7b0lmSqqibG+ZorScv9b7nvYP9b7/9c+izvHAI2J9mU5AwGX8xOjtSZBHZ2x5cDd9fgI8QksKN7umcTsBn49HiaLkmarzln+lV1MsnVwB3AGuCmqjqcZA8wVVWTwI3AzUmmGczwd3T3Hk5yK/BF4CTwjqp6eon6Ikmaw5xr+itVkl3dslGTWu5/y30H+996/+eyakNfkvRsbsMgSQ1ZlaE/17YRq02Sm5I8muQLQ2UvTXJnki91/33JcrZxqSQ5P8knktyf5HCSd3blrfT/rCSfTvLZrv8f6Mo3dVuifKnbIuWM5W7rUkmyJsl9Sf6hO2+m7wux6kK/57YRq82fM9jmYthu4K6q2gzc1Z2vRieBd1XVK4DXA+/o/n+30v+ngDdV1Q8Drwa2JXk9g61Q9nb9P85gq5TV6p3A/UPnLfV93lZd6NNv24hVpar+lcFTU8OGt8bYD7zttDbqNKmqR6rqM93x/zD4419PO/2vqnqiO31B96+ANzHYEgVWcf+TbAAuAz7anYdG+r5QqzH03fph4Puq6hEYBCPwvcvcniXX7e76I8A9NNT/bnnj34FHgTuBLwNf77ZEgdX9N/Bh4D3At7vzc2mn7wuyGkM/s5T5iNIql+TFwMeB36yqbyx3e06nqnq6ql7N4BfvW4FXzFbt9LZq6SV5C/BoVd07XDxL1VXX98Xosw3DStNr64cGfDXJy6rqkSQvYzALXJWSvIBB4P9lVf1NV9xM/59RVV9P8s8Mvts4J8nabsa7Wv8Gfgx4a5KfAc4CvofBzL+Fvi/Yapzp99k2ogXDW2PsBP5+GduyZLo13BuB+6vqQ0OXWun/uiTndMcvBH6Kwfcan2CwJQqs0v5X1TVVtaGqNjL4O7+7qn6BBvq+GKvyx1ndyP9hvrNtxB8sc5OWVJK/Bt7IYHfBrwLvA/4OuBW4APgKcEVVjX7Zu+IleQPwb8Dn+c667u8yWNdvof8XMfiycg2DSdytVbUnycsZPMTwUuA+4Ber6qnla+nSSvJG4N1V9ZbW+j5fqzL0JUmzW43LO5KkUzD0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqyP8Bst/13MP2gdUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#绘制PCA方差\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "plt.bar(range(len(pca.explained_variance_ratio_)),pca.explained_variance_ratio_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3   采用train_test_split，从将数据集中随机抽取10000条记录（原始数据集太大，剩余数据抛弃，此部分SVM作业已经完成）。对这部分数据进行PCA降维，保留85%的能量。（20分）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(10000, 95)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/liu/anaconda3/lib/python3.7/site-packages/sklearn/model_selection/_split.py:2026: FutureWarning: From version 0.21, test_size will always complement train_size unless both are specified.\n",
      "  FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from matplotlib import pyplot as plt\n",
    "train=pd.read_csv(\"Otto_FE_train_tfidf.csv\")\n",
    "#按照题目采用train_test_split抽取数据\n",
    "from sklearn.model_selection import train_test_split\n",
    "train,train_drop=train_test_split(train,train_size=10000,random_state=0)\n",
    "#train=train.sample(n=10000)\n",
    "print(train.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<BarContainer object of 48 artists>"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD8CAYAAACb4nSYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAEAhJREFUeJzt3X+s3Xddx/Hny5ZtCHGDrhhcN1uyaigBUWohEZUw3TqHK8YtdqDWZKaaMIMBxM7EARWTzRiKiTOxYdM51G0Z/mhcdVko/ggho3eMX6U2XOZk1y6so2U4zVY63v5xvgvHs9vd77333J72fp6P5Kbf7+f7+Z7z+aS5r+/nfM73+7mpKiRJbfiuSTdAknTqGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0JekhqycdANGnX/++bV27dpJN0OSzigPPPDA41W1eq56p13or127lqmpqUk3Q5LOKEn+s089p3ckqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhp90TuYu1dsc9zyl7+MYrJtASSTr9ONKXpIYY+pLUkF6hn2RzkkNJppPsmOX4TyT5TJITSa4aObYtyZe7n23jargkaf7mDP0kK4CbgcuBDcA1STaMVPsq8CvAX42c+1LgfcDrgU3A+5K8ZPHNliQtRJ+R/iZguqoeqqrjwB3AluEKVfVwVX0e+PbIuZcB91XV0ao6BtwHbB5DuyVJC9An9C8AHhnan+nK+uh1bpLtSaaSTB05cqTnS0uS5qtP6GeWsur5+r3OrardVbWxqjauXj3nH36RJC1Qn9CfAS4c2l8DHO75+os5V5I0Zn1Cfz+wPsm6JGcBW4E9PV//XuDSJC/pvsC9tCuTJE3AnKFfVSeA6xiE9UHgrqo6kGRnkisBkvxokhngauBPkxzozj0K/B6DC8d+YGdXJkmagF7LMFTVXmDvSNkNQ9v7GUzdzHburcCti2ijJGlMfCJXkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SG9Ar9JJuTHEoynWTHLMfPTnJnd/z+JGu78hckuS3JF5IcTHL9eJsvSZqPOUM/yQrgZuByYANwTZINI9WuBY5V1cXALuCmrvxq4OyqejXwOuDXnr0gSJJOvT4j/U3AdFU9VFXHgTuALSN1tgC3ddt3A5ckCVDAi5KsBF4IHAe+OZaWS5LmrU/oXwA8MrQ/05XNWqeqTgBPAKsYXAD+B3gU+Crwh1V1dPQNkmxPMpVk6siRI/PuhCSpnz6hn1nKqmedTcAzwPcB64B3J3nFcypW7a6qjVW1cfXq1T2aJElaiD6hPwNcOLS/Bjh8sjrdVM65wFHgbcA/VdW3quox4JPAxsU2WpK0MH1Cfz+wPsm6JGcBW4E9I3X2ANu67auAfVVVDKZ03pyBFwFvAP59PE2XJM3XnKHfzdFfB9wLHATuqqoDSXYmubKrdguwKsk08C7g2ds6bwZeDHyRwcXjz6rq82PugySpp5V9KlXVXmDvSNkNQ9tPMbg9c/S8J2crlyRNhk/kSlJDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhrS6+Gs5WDtjnueU/bwjVdMoCWSNDmO9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkN6hX6SzUkOJZlOsmOW42cnubM7fn+StUPHXpPkU0kOJPlCknPG13xJ0nzMGfpJVgA3A5cDG4BrkmwYqXYtcKyqLgZ2ATd1564EPgr8elW9CngT8K2xtV6SNC99RvqbgOmqeqiqjgN3AFtG6mwBbuu27wYuSRLgUuDzVfU5gKr6elU9M56mS5Lmq0/oXwA8MrQ/05XNWqeqTgBPAKuAHwAqyb1JPpPkvbO9QZLtSaaSTB05cmS+fZAk9dQn9DNLWfWssxJ4I/D27t+fS3LJcypW7a6qjVW1cfXq1T2aJElaiD6hPwNcOLS/Bjh8sjrdPP65wNGu/F+q6vGq+l9gL/Aji220JGlh+oT+fmB9knVJzgK2AntG6uwBtnXbVwH7qqqAe4HXJPnu7mLwk8CXxtN0SdJ8rZyrQlWdSHIdgwBfAdxaVQeS7ASmqmoPcAtwe5JpBiP8rd25x5J8iMGFo4C9VXXPEvVFkjSHOUMfoKr2MpiaGS67YWj7KeDqk5z7UQa3bUqSJswnciWpIYa+JDXE0JekhvSa01/O1u547vfKD994xQRaIklLz5G+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNaf4+/efjPfySlhtH+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0JekhvhHVBbAP64i6UzlSF+SGmLoS1JDeoV+ks1JDiWZTrJjluNnJ7mzO35/krUjxy9K8mSS94yn2ZKkhZgz9JOsAG4GLgc2ANck2TBS7VrgWFVdDOwCbho5vgv4x8U3V5K0GH1G+puA6ap6qKqOA3cAW0bqbAFu67bvBi5JEoAkbwUeAg6Mp8mSpIXqE/oXAI8M7c90ZbPWqaoTwBPAqiQvAn4b+MDimypJWqw+oZ9ZyqpnnQ8Au6rqyed9g2R7kqkkU0eOHOnRJEnSQvS5T38GuHBofw1w+CR1ZpKsBM4FjgKvB65K8gfAecC3kzxVVX88fHJV7QZ2A2zcuHH0giJJGpM+ob8fWJ9kHfBfwFbgbSN19gDbgE8BVwH7qqqAH3+2QpL3A0+OBr4k6dSZM/Sr6kSS64B7gRXArVV1IMlOYKqq9gC3ALcnmWYwwt+6lI2WJC1Mr2UYqmovsHek7Iah7aeAq+d4jfcvoH2SpDHyiVxJaogLro2RC7FJOt050pekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kN8ZbNU8TbOSWdDhzpS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhriw1kT5kNbkk4lR/qS1BBDX5Ia4vTOaWq2aR9w6kfS4jjSl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktSQXsswJNkM/BGwAvhIVd04cvxs4C+A1wFfB36hqh5O8tPAjcBZwHHgt6pq3xjb3yRX5pS0UHOO9JOsAG4GLgc2ANck2TBS7VrgWFVdDOwCburKHwd+tqpeDWwDbh9XwyVJ89dnemcTMF1VD1XVceAOYMtInS3Abd323cAlSVJVD1bV4a78AHBO96lAkjQBfaZ3LgAeGdqfAV5/sjpVdSLJE8AqBiP9Z/088GBVPb3w5ur5OO0jaS59Qj+zlNV86iR5FYMpn0tnfYNkO7Ad4KKLLurRJEnSQvQJ/RngwqH9NcDhk9SZSbISOBc4CpBkDfC3wC9X1Vdme4Oq2g3sBti4cePoBUVj4KcASdBvTn8/sD7JuiRnAVuBPSN19jD4ohbgKmBfVVWS84B7gOur6pPjarQkaWHmDP2qOgFcB9wLHATuqqoDSXYmubKrdguwKsk08C5gR1d+HXAx8LtJPtv9vGzsvZAk9dLrPv2q2gvsHSm7YWj7KeDqWc77IPDBRbZRS8hpH6kt/o1czcqLgbQ8uQyDJDXE0Jekhji9o3lz6kc6cznSl6SGGPqS1BCndzQ2TvtIpz9H+pLUEEf6WnKzfQIAPwVIk+BIX5Ia4khfE+X3ANKpZejrtPR8FwMvFNLCGfpaNk52MfAiIX2Hc/qS1BBH+mrafD8d+KlBZzpDXxoTLwg6Exj60hKbz6eGZ49JS8XQl05DTi9pqRj60jLhhUJ9GPpSoxbyLIQXkDOfoS9p0XyY7sxh6EuaCC8Uk2HoSzpjLGTayWcx/j9DX5J6GscF5Nljk2LoS9IETOoThWvvSFJDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIb0Cv0km5McSjKdZMcsx89Ocmd3/P4ka4eOXd+VH0py2fiaLkmarzlDP8kK4GbgcmADcE2SDSPVrgWOVdXFwC7gpu7cDcBW4FXAZuBPuteTJE1An5H+JmC6qh6qquPAHcCWkTpbgNu67buBS5KkK7+jqp6uqv8AprvXkyRNQJ/QvwB4ZGh/piubtU5VnQCeAFb1PFeSdIqkqp6/QnI1cFlV/Wq3/0vApqr6jaE6B7o6M93+VxiM6HcCn6qqj3bltwB7q+pjI++xHdje7f4gcGgMfTsfeHwMr3Omarn/Lfcd7H+r/f/+qlo9V6U+a+/MABcO7a8BDp+kzkySlcC5wNGe51JVu4HdPdrSW5Kpqto4ztc8k7Tc/5b7Dva/9f7Ppc/0zn5gfZJ1Sc5i8MXsnpE6e4Bt3fZVwL4afITYA2zt7u5ZB6wHPj2epkuS5mvOkX5VnUhyHXAvsAK4taoOJNkJTFXVHuAW4PYk0wxG+Fu7cw8kuQv4EnACeEdVPbNEfZEkzWHOOf0zVZLt3bRRk1ruf8t9B/vfev/nsmxDX5L0XC7DIEkNWZahP9eyEctNkluTPJbki0NlL01yX5Ivd/++ZJJtXCpJLkzyiSQHkxxI8s6uvJX+n5Pk00k+1/X/A135um5JlC93S6ScNem2LpUkK5I8mOQfuv1m+r4Qyy70ey4bsdz8OYNlLobtAD5eVeuBj3f7y9EJ4N1V9UrgDcA7uv/vVvr/NPDmqvoh4LXA5iRvYLAUyq6u/8cYLJWyXL0TODi031Lf523ZhT79lo1YVqrqXxncNTVseGmM24C3ntJGnSJV9WhVfabb/m8Gv/wX0E7/q6qe7HZf0P0U8GYGS6LAMu5/kjXAFcBHuv3QSN8XajmGvks/DHxvVT0Kg2AEXjbh9iy5bnXXHwbup6H+d9MbnwUeA+4DvgJ8o1sSBZb378CHgfcC3+72V9FO3xdkOYZ+ZinzFqVlLsmLgY8Bv1lV35x0e06lqnqmql7L4In3TcArZ6t2alu19JK8BXisqh4YLp6l6rLr+2L0WYbhTNNr6YcGfC3Jy6vq0SQvZzAKXJaSvIBB4P9lVf1NV9xM/59VVd9I8s8Mvts4L8nKbsS7XH8Hfgy4MsnPAOcA38Ng5N9C3xdsOY70+ywb0YLhpTG2AX8/wbYsmW4O9xbgYFV9aOhQK/1fneS8bvuFwE8x+F7jEwyWRIFl2v+qur6q1lTVWga/5/uq6u000PfFWJYPZ3VX/g/znWUjfn/CTVpSSf4aeBOD1QW/BrwP+DvgLuAi4KvA1VU1+mXvGS/JG4F/A77Ad+Z1f4fBvH4L/X8Ngy8rVzAYxN1VVTuTvILBTQwvBR4EfrGqnp5cS5dWkjcB76mqt7TW9/lalqEvSZrdcpzekSSdhKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JD/g9Rz+LjSpI06gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "y_train=train['target']\n",
    "X_train=train.drop([\"id\",\"target\"],axis=1)\n",
    "\n",
    "train_id=train['id']\n",
    "#PCA降维\n",
    "pca=PCA(n_components=0.85)\n",
    "pca.fit(X_train)\n",
    "\n",
    "X_train_pca=pca.transform(X_train)\n",
    "\n",
    "#绘制PCA方差\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "plt.bar(range(len(pca.explained_variance_ratio_)),pca.explained_variance_ratio_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4   对3中得到的数据（对降维后的数据），训练RBF核SVM，并对超参数（C和gamma）进行超参数调优。结果和用原始数据的情况比较（SVM部分作业结果）。（30分）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/liu/anaconda3/lib/python3.7/site-packages/sklearn/model_selection/_split.py:2026: FutureWarning: From version 0.21, test_size will always complement train_size unless both are specified.\n",
      "  FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "feat_names=X_train.columns\n",
    "#数据用稀疏矩阵表示，模型训练变快\n",
    "from scipy.sparse import csr_matrix\n",
    "X_train=csr_matrix(X_train_pca)\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train_part,X_val,y_train_part,y_val=train_test_split(X_train,y_train,train_size=0.8,random_state=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "#超参数调优，正则参数C,RBF核函数的宽度\n",
    "from sklearn.svm import SVC\n",
    "def fit_grid_point_RBF(C,gamma,X_train,y_train,X_val,y_val):\n",
    "    SVC3=SVC(C=C,kernel='rbf',gamma=gamma)\n",
    "    SVC3=SVC3.fit(X_train,y_train)\n",
    "    accuracy_train=SVC3.score(X_train,y_train)\n",
    "    accuracy=SVC3.score(X_val,y_val)\n",
    "    print(\"C={} and gamma={}:,accuracy_train={}，accuracy_val={}\" .format(C,gamma,accuracy_train,accuracy))\n",
    "    return accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C=0.1 and gamma=0.1:,accuracy_train=0.6995，accuracy_val=0.7055\n",
      "C=0.1 and gamma=1.0:,accuracy_train=0.736375，accuracy_val=0.7405\n",
      "C=0.1 and gamma=10.0:,accuracy_train=0.55225，accuracy_val=0.5505\n",
      "C=1.0 and gamma=0.1:,accuracy_train=0.73925，accuracy_val=0.7455\n",
      "C=1.0 and gamma=1.0:,accuracy_train=0.799，accuracy_val=0.7785\n",
      "C=1.0 and gamma=10.0:,accuracy_train=0.95625，accuracy_val=0.7445\n",
      "C=10.0 and gamma=0.1:,accuracy_train=0.768875，accuracy_val=0.765\n",
      "C=10.0 and gamma=1.0:,accuracy_train=0.914，accuracy_val=0.7875\n",
      "C=10.0 and gamma=10.0:,accuracy_train=0.996375，accuracy_val=0.7475\n",
      "C=100.0 and gamma=0.1:,accuracy_train=0.825625，accuracy_val=0.7845\n",
      "C=100.0 and gamma=1.0:,accuracy_train=0.980625，accuracy_val=0.7595\n",
      "C=100.0 and gamma=10.0:,accuracy_train=1.0，accuracy_val=0.7485\n",
      "C=1000.0 and gamma=0.1:,accuracy_train=0.904125，accuracy_val=0.777\n",
      "C=1000.0 and gamma=1.0:,accuracy_train=0.99625，accuracy_val=0.7605\n",
      "C=1000.0 and gamma=10.0:,accuracy_train=1.0，accuracy_val=0.7485\n"
     ]
    }
   ],
   "source": [
    "accuracy_s=np.matrix(np.zeros(shape=(5,3)),float)\n",
    "\n",
    "#需要调整的参数\n",
    "gamma_s=np.logspace(-1,1,3)\n",
    "\n",
    "C_s=np.logspace(-1,3,5)#吧10的-1次方到10的3次方分成5份\n",
    "\n",
    "\n",
    "for i, oneC in enumerate(C_s):\n",
    "    for j,gamma in enumerate(gamma_s):\n",
    "        tmp=fit_grid_point_RBF(oneC,gamma,X_train_part,y_train_part,X_val,y_val)\n",
    "        accuracy_s[i,j]=tmp\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl8VPX1+P/XyUZCCFlIQkIChiWQsIqExV0QBK0F+Vbr9mn1o3Vpq21ta13aImD7q9rF1k9pq1Vb2w8V+xGr1FqpCmhVVIKiln0RJSGBECCEJeuc3x/3Jpkkk2QIuZks5/l4zCNz77zvzJmBzMl9v9/3vEVVMcYYY1oTFuoAjDHGdH2WLIwxxrTJkoUxxpg2WbIwxhjTJksWxhhj2mTJwhhjTJssWRhjjGmTJQtjjDFtsmRhjDGmTRGhDqCjJCcna1ZWVqjDMMaYbmX9+vUHVDWlrXY9JllkZWWRn58f6jCMMaZbEZFPg2ln3VDGGGPa5GmyEJE5IrJVRHaIyN0BHh8iIqtF5AMR+UhELvF77B73uK0iMtvLOI0xxrTOs24oEQkHlgCzgAJgnYisUNVNfs1+APxVVX8rIqOBl4As9/5VwBhgEPCqiIxU1Vqv4jXGGNMyL88spgA7VHWXqlYBy4B5Tdoo0N+9Hw/sde/PA5apaqWqfgLscJ/PGGNMCHiZLDKAPX7bBe4+fwuB/xKRApyzittP4lhE5GYRyReR/JKSko6K2xhjTBNeJgsJsK/pSktXA39U1UzgEuDPIhIW5LGo6mOqmqeqeSkpbc78MsYY005eTp0tAAb7bWfS0M1U50ZgDoCqrhWRaCA5yGONMcZ0Ei+TxTogW0SGAoU4A9bXNGnzGXAh8EcRyQWigRJgBfAXEfkFzgB3NvCeh7EaY0Kk1qfsL69g7+ETFB52fkaECZmJMWQm9iUzMYb4mEhEAnU4mM7iWbJQ1RoRuQ1YCYQDT6rqRhFZDOSr6grgO8DvReQOnG6m69VZFHyjiPwV2ATUAF+3mVDGdE/lFdXsPVyXDE6wt/5WQeHhExQfqaDW16yXuZHYqPD6xJGZGEOGXyLJTOxLYl9LJl4T57u5+8vLy1O7gtu0yecDXw1orfPTV+vcGm3XgPoabwfap7UNxzd6zgCv0S8VUnIgcSiE95jCCdTU+thXXlmfAAr9EkHddnlFTaNjIsKE9IRo0uNjyEiIYVBCNIMSYhiU4Gynx0fj88GeQ8cpOHSCAvdn4eETzvbB45RXNn7OmMjw+kSSmdjXTSYNCWVAbJQlkxaIyHpVzWurXc/5X2s6R9Vx2P1vOF7a5Mu06RduEF/AWtv4+Ba/gAN9IQfzJR3gS7v5PInOFRYJA0ZAykgneSSPhJRRMCAbIqNDG1sTqsqRihq/M4GGbqK6W/GRCpqeFCT2jWRQQgyDk/oybdiAZskguV8fwsPa/uKO7xvP2Iz4gI+Vnaim0C+ROMnEuf/+Z4cpO1HdqH10ZBgZCQ3Jo/GZSQwp/fpYMmmDJQvTtsqjsP1fsOkF52f18eCPlTCQcAiLgLBw59ZoOyLAvhbaRPRp2JbwhrZhEYH3BdyOgLCwJtuB9rUQh4Q12Q5vHGPA1w2DI3uhZCsc2Or8LP4PbP67k0DrPqeE05wEUp9IRjn3+8R58s9aVeNj35GKRl1DhYcrKCprODs42uQv+KjwMNITohkUH8OZw5PJ8EsEg9yzhL5R3n+txMdEEh8TyehB/QM+Xl5R7ZyJHGx+ZvJRwWEOHW+cTKIiwshMaJ5E6s5OUvr1ISyIBNeTWTeUCayiDLatdBLEjlehpgJiUyH3884taWgbX9J1SaB3/4K1qroCDu50kod/IindAbVVDe36Z7hnIE0SSeyAFp9aVTl8vLrxGEFZ48Swv7ySpr/+A2Kj6r/0684E/BNBcmzP+NI8VlnjJg+/MxO/M5XSY1WN2keFhzEoIbrFcZPUuOigzpa6omC7oSxZmAYnDsHWfzoJYucq5wsrbpCTHEbPgyHTnARgvFVbA4d2u8ljC5Rsc+9vg+pjDc1iBnA0bjgHYoZSED6Y7ZrBRxUD2Vjel71llZyobjwnJCoirGGMID6mSTJwkkN0pP37Ahyvcrrf9riJpODQcTeZOLcDRysbtY8MF9LjG5+NON1eMWQm9WVgXB8iwrtm3VZLFiY4x0phy4tOgvjkdadvP36wkxxGz4OMPKeLxnQaVeXgsar62UJ1ZwJFh49RcbCA2LIdpFTsZoQUMiKskGwpJEEaksiJsFhKY7I43n84vuRR9EnPJX7IWBLThyM9aHA9lCqqaxsG3Ou6ufzu7y9vnEwiwoS0+OhGg+7+Yyjp8dEhSyY2wG1adnS/01++6QXY/aYzAJyYBWfeBqPnwqAzrPvIQxXVtRSVBZ5KWrevssbX6JjoyDD3TGAQyYOG0z8hhoiEaKoTYjgcH010RDnRh7dDyVZiSraSeWArlLwLRSvgY/dJIqIhOdsdC3G7tJJHQdIwiIjq/A+iG4uODGd4Sj+Gp/QL+Hjdv3GB34yuujOTN7cfYF95RaMuwPAwIa1/dONZXAkN99MTookM8ZmJnVn0Fkf2NiSIT98G1JmBU3cGkTbOEkQHqqypZcf+o2wpKmfrvnI+Kz3OXnfg+MDRxv3hIpAa16fRjKFB8Y0Hjtt9HcGJQ3Bgu9ud5Tc2cvizhjZhEU7CSBnVOJEMyIaovqf4SZhAqmp8FJU1PjPxPztpOsssTPBLJn7jJgnumUlCNH0i2teFaN1QxvlC2LTCSRAF7gXwqaMbEkRKjiWIU6SqFJVVsKX4CJuLytlaXM6W4iPsLDlWf6FZVEQYpyX19UsGjaeSDuwfTVREJ//VWHXMTSJ+A+slW+HgLudMEwCBhCFOEmmaSKIDT2k1HaOqxkdxWQUFhxsSif/ZSVHZiUbJZMyg/vzjG+e267WsG6q3OrirIUHsfd/ZlzYeZvwAcuc5v+imXY5V1rBtXzlbisvZUnSEze7PI34XnWUkxJCbHsdFo9PISY8jJ60/WQP6dr3BzahYGHS6c/NXUxV4htau16HWrx8+Lr2FGVrJ9gdIB4iKCGPIgL4MGRD4zK661kkmdeMmnfHHhp1Z9AQHtsOm550EUex2UA86wz2DmOt0MZig+XzKZweP158tbCk+wpbicj4tbbi+JDYqnJz0/uSkxdX/HJUWR//oyBBG7iFfrTtDa1vDDK2SLc521dGGdjGJfhcb+iWS/hmWRLoo64bqyVRh/2YnOWxeAfvdxQcHT4XcuU6CSBgS2hi7icPHq+rPFLYUO2cNW4vL66edhglkJceSm9afUWlx5KTFkZven4yEmB5xvcEpU3UvONzSPJGcONjQLqpfw9Xq9V1ao5yJFTYdO6QsWfQ0qs5Zw6YXnFvpdkDgtLOcM4jcz0P/QaGOssuqrvWxq+RY/VlCXXIoKquob5PYN5KctP7kpMeR6/7MTo0jJsq+zNrl2IHmA+sl26Dcb7WB8D5u+ZNRjRNJv4HODK3wKKdEik3f9oyNWfQEqs64Q12COLTbKQuRdS5M+yrkXApxA0MdZZeiqpSUV7pnCUfYUlTO5uJyduwvp7rW+cMoMlwYntKPacMGNOpGSo2z+kAdKjYZYs+BrHMa768oaz5Da+/7sPFvtFi7KyzCSRzhkU6Cqb/vJpS6xOK/L2Bb92dEnwBto9z2AfbXP38rbXt4xQJLFl2NzwcF65zupU0vQNke5xdl6Plwzrch53POL6GhorqW7fuOstlNCnVnDQf9SjWk9Y8mJz2O80emuIkhjmHJ/Tp/9pFpEB0PmXnOzV/VcafUSclWpwurtsq51VQ13K+t9rsfYF9NlVPLrFG7amdw3r+tryZwbKdEAiQi/8QTKOFEuu2DTHQttY1Jaj5ZoYNZsugKfLXw2TsNYxDlRc5/guEzYPq9MOpiZ+Cwl1JVCg6daNR9tLn4CLsPHKufPhgdGcaotP7Myh1YPwspJy2OxFi72KzbiOoL6eOdm9d8PvBVQ01lgMRS5SaXAPtbbN9K8mq2v9qZFNDs+KbxVLb9Pupk5MFNr3n3eWHJInRqa+DTN90E8SIc2+9cYTtipjMGMXJ2r5zLXl5RzdZip+toq3vGsLW4vNH6BUOS+pKTFsel4weR63YjDUnq220LuZkQCAuDsD7OX+pdlarzh2Srycvd7oTy9pYsOlNttVN/qS5BnDgIkX0h+yInQWRfBH0Clw/oaWp9yu7SY/XdR3VTVAsOnahvExcdQW5af+afkVE/8DxyYBz9+th/W9MLiDgLZYVHAKG/kt7T3zoRmQP8CmdZ1cdV9YEmjz8MTHc3+wKpqprgPlZLQ1Wbz1R1rpexeqamEnaudhLE1n84g3tRcTBqjpMghl/Y40sqHDxW1egiti3F5WzbV15f/yg8TBiWHMvEIYlcPWVI/aDzoPhoG3A2povwLFmISDiwBJgFFADrRGSFqm6qa6Oqd/i1vx2Y6PcUJ1TV2xEbr1SfcNaA2LQCtr0MlUecLqVRn3OugRg2vcutitYRKmtq2bm/YXrq5qIjbC0ub1SBM7lfH3LT4/jStNPqZyGNSO1npbGN6eK8PLOYAuxQ1V0AIrIMmAdsaqH91cB9HsbjrcqjsOMV5wxi27+cdQdiktyrqC+Doef1mMqeqkrxkQp3WqqTELYUlbOz5Cg1fvWQslP7cW52CrnugPOotDhS4rpwH7ExpkVeJosMYI/fdgEwNVBDETkNGAqs8tsdLSL5QA3wgKo+71Wg7VZxxF1N7nm/1eRSYMKVTpI47Ry3v7H7KztRzb+3l7Bqy37e2FbSqHJqRkIMOWlxzBydSk5af3LT48gaENv16iEZY9rNy2+yQJ3NLV0ufhXwrKr6L+01RFX3isgwYJWIfKyqOxu9gMjNwM0AQ4Z0UnmL+tXkVsDO19zV5NLhjC+7q8md2SPKF6gqO/YfZdWW/azasp/1nx6ixqck9I3k/JEp5J2WyCj3bCE+pofWQzLG1PMyWRQAg/22M4G9LbS9Cvi6/w5V3ev+3CUia3DGM3Y2afMY8Bg45T46JOpAjpU6g9ObXoBda5wLevpnwuSbnASROblHlCOoqK5l7a5SVrsJom5mUk5aHDefN4wZOamcPjjBzhiM6YW8TBbrgGwRGQoU4iSEa5o2EpFRQCKw1m9fInBcVStFJBk4G3jIw1iba3E1ua87pb4zesZqcnsPn2DVlv2s3rKft3YeoKLaR0xkOGePGMBXLxjO9FGpDEqICXWYxpgQ8yxZqGqNiNwGrMSZOvukqm4UkcVAvqqucJteDSzTxhUNc4FHRcQHhOGMWbQ0MN5xjhT5rSb3Fs5qciPgnG+5q8mN7/YJoqbWxwd7DtcniC3F5QAMTorhyrzBTM9JZdqwATY7yRjTiFWdrSiDD/7XSRB73nX2peQ2rCaXmtvtE8ShY1W8vs0ZnH59WwllJ6qJCBPyshKZkZPKjJxUhqf0s2sajOmFrOpssHy18K8fwsDRMP0HznUQKaNCHdUpUVU2F5Wzeqsz9vDBZ4fwKST3i2Jm7kBm5KRy7sjknrtQjzGmw1my6JsEd2yE/umhjuSUHK+q4a0dpazasp81W/fXr9MwLiOe22ZkMyMnlfEZ8bZgjzGmXSxZQLdNFJ+VHmfVln2s2lrCO7tKqarx0a9PBOeMSOaOmalcMCqF1P4970pxY0zns2TRjVTX+li3+2D91NadJccAGJYSy5enncaMnFTyspJsrQZjTIezZNHFlZRXsmbrflZv3c+/tx2gvLKGqPAwpg5L4r+mncb0UalkJceGOkxjTA9nyaKL8fmUjwvLnKmtW/fzUUEZAAP79+HSCelMH5XK2SOSibUy3caYTmTfOF1AeUU1/95+wB2cLuHA0UpEYOLgBL570Uim56QyOr2/TW01xoSMJYsQUFV2lhyrH3tYt/sgNT6lf3QE549KZUZOCuePTCXJlgQ1xnQRliw6SUV1Le9+0jA4/dnB4wCMGhjHV8516i6dMcTqLhljuiZLFh4qLquor9r61o4DnKiupU9EGGePSOam84YxfVQKmYk9e5U8Y0zPYMmiA9X6lA17DrkJooTNRUcAZ72HyydlMiMnlTOHW90lY0z3Y8niFB0+7tRdWu3WXTp0vJrwMGHSaYncfXEOM3JSyU61ukvGmO7NksVJUlW27iuvr9q6/lOn7lJSbBTTR6UyPSeV87JTiO9rdZeMMT2HJYsgnKiq5e2dDVNbCw87iwKNGdSfr08fwfScVCZkJhBudZeMMT2UJYsW7Dl4vL5q69qdpVTW+OgbFc45I5K5fYaTIAZa3SVjTC9hycJVXetj/aeH6qe2bt9/FICsAX25ZuoQZuSkMmVoEn0ibHDaGNP79Ppksf9IBYte3MQb20oor6ghMlyYOnQAV01xEsRQq7tkjDHeJgsRmQP8CmdZ1cdV9YEmjz8MTHc3+wKpqprgPnYd8AP3sR+p6lNexNg/JpL/FJZxydh0puekck52Mv2s7pIxxjTi2bKqIhIObANmAQXAOuDqltbSFpHbgYmqeoOIJAH5QB6gwHpgkqoeaun12r2sKs4MJ5vaaozpjYJdVtXL2hJTgB2quktVq4BlwLxW2l8NPO3enw28oqoH3QTxCjDHq0AtURhjTOu8TBYZwB6/7QJ3XzMichowFFh1sscaY4zxnpfJItCf6y31eV0FPKuqtSdzrIjcLCL5IpJfUlLSzjCNMca0xctkUQAM9tvOBPa20PYqGrqggj5WVR9T1TxVzUtJSTnFcI0xxrTEy2SxDsgWkaEiEoWTEFY0bSQio4BEYK3f7pXARSKSKCKJwEXuPmOMMSHg2RxRVa0RkdtwvuTDgSdVdaOILAbyVbUucVwNLFO/aVmqelBE7sdJOACLVfWgV7EaY4xpnWdTZzvbqUydNcaY3qorTJ01xhjTQ1iyMMYY0yZLFsYYY9pkycIYY0ybLFkYY4xpkyULY4wxbbJkYYwxpk2WLIwxxrTJkoUxxpg2WbIwxhjTJksWxhhj2mTJwhhjTJssWRhjjGmTZyXKjenNVJXDlYcpOlZE8bFiqnxVREok4WHhhEs4EWER9bdwCSc8LJwIabyv6f1wCScyzHmOMLG/80znsmRhTDtU1lay79g+io4V1d+KjxVTdLThfkVthWevL0hDwpEIJ9m0kmSa7QsLbz15tZDQIsMiT+l5Wzqu6ePhYeGefXY9kSBEhUd5+hqWLIxpQlU5WHHQ+fJvIRmUVpQ2Oy45Jpn02HSyE7M5L/M80mPTSY9NJy02jeiIaGp8NdRoDTW+Gmp9tfXbdfdrNfC+al+1s123v8lzBDrOv43/4zW+GipqKprtaxZPgOcyXdf45PEs/dxST1/DkoXpdSpqKuoTQaCEUHysmMraykbHxETEkBabRnpsOqOSRtXfr7sNjB3o+V92oaSq+NQXMMk0SmjtSF6BEpVPfaF+y91Kat9Uz1/D02QhInOAX+Esq/q4qj4QoM0XgYWAAh+q6jXu/lrgY7fZZ6o618tYTc/gUx8HKw7WnwE0TQjFx4o5WNF4hV5BSIlJIa1fGjlJOUwfPL1ZMojvE4+IhOhdhZ6ION1DhPfopGha5lmyEJFwYAkwCygA1onIClXd5NcmG7gHOFtVD4mIf3o8oaqnexWf6Z6OVx+n+HgxxUcDdBG5P6t91Y2OiYmIYVDsINL6pTF6wOhG3UPpsekM7DuQyPDIEL0jY7oHL88spgA7VHUXgIgsA+YBm/za3AQsUdVDAKq638N4TBfnUx8HThxoSABHi5slg8OVhxsdEyZhpMSkkB6bztgBY5l52sxGZwRpsWn0j+rfq88KjOkIXiaLDGCP33YBMLVJm5EAIvIWTlfVQlV92X0sWkTygRrgAVV93sNYTSc4Vn2s8RjB0cZdRPuO72s2kBobGVv/xT8ueRzp/dIbdRGl9E0hMszOCozxmpfJItCfchrg9bOBC4BM4N8iMlZVDwNDVHWviAwDVonIx6q6s9ELiNwM3AwwZMiQjo7fnIRaXy0lJ0paTQZHqo40OiZcwkntm0p6bDoTUiY0nBH4JYS4qLgQvSNjjD8vk0UBMNhvOxPYG6DNO6paDXwiIltxksc6Vd0LoKq7RGQNMBFolCxU9THgMYC8vLymich45I2CN/hg/weNEsK+4/uo1dpG7eKi4uoTwOmppzdKBumx6STHJBMRZhPyjOkOvPxNXQdki8hQoBC4CrimSZvngauBP4pIMk631C4RSQSOq2qlu/9s4CEPYzVBWle8jq+/9nUiJIKBsQNJi03jjIFnNBowrrvfL6pfqMM1xnQQz5KFqtaIyG3ASpzxiCdVdaOILAbyVXWF+9hFIrIJqAXuVNVSETkLeFREfDj1qx7wn0VlQqOytpLFaxeT0S+D5+Y+R9/IvqEOyRjTSYJKFiKyHHgS+Kdq8FfLqOpLwEtN9i3wu6/At92bf5u3gXHBvo7pHI9//Di7j+zm0ZmPWqIwppcJthrZb3G6kLaLyAMikuNhTKYL2nl4J49//DifG/Y5zso4K9ThGGM6WVDJQlVfVdVrgTOA3cArIvK2iPy3iNi8xR7Opz4Wr11MbGQsd+bdGepwjDEhEHSdYxEZAFwPfAX4AKeMxxnAK55EZrqM5duX8/7+9/nOpO8wIGZAqMMxxoRAsGMWzwE5wJ+Bz6tqkfvQM+6Fc6aHKjlewsP5DzM5bTKXjbgs1OEYY0Ik2NlQv1bVVYEeUNW8DozHdDEPrnuQytpKFkxbYCUzjOnFgu2GyhWRhLoNEUkUka95FJPpIt4oeIOVu1dy8/ibyYrPCnU4xpgQCjZZ3OSW4ADALfx3kzchma7gePVxfvTOjxgeP5wbxt4Q6nCMMSEWbDdUmIiIe11EXflxK2rfgy3ZsISiY0X86eI/WfluY0zQyWIl8FcR+R1OMcBbgZdbP8R0VxtLN/K/m/+XL478IhNTJ4Y6HGNMFxBssrgLuAX4Kk412X8Bj3sVlAmdGl8Ni95eRFJ0Et+c9M1Qh2OM6SKCShZuiY/fujfTgy3dvJTNBzfz8/N/Tv+o/qEOxxjTRQR7nUU28BNgNBBdt19Vh3kUlwmBwqOFLNmwhPMzz2fWabNCHY4xpgsJdjbUH3DOKmqA6cCfcC7QMz2EqvKjd34EwPenft+uqTDGNBJssohR1dcAUdVPVXUhMMO7sExnW7l7JW8WvsntE28nvV96qMMxxnQxwQ5wV4hIGE7V2dtwFjNK9S4s05nKKst44L0HGDNgDNfkNF2fyhhjgj+z+BbQF/gGMAn4L+A6r4Iynevh9Q9zuPIw9515H+Fh4aEOxxjTBbV5ZuFegPdFVb0TOAr8t+dRmU6zft96lm9fzvVjrid3QG6owzHGdFFtnlmoai0wSWzEs8epqq1i0dpFDIodxFcnfDXU4RhjurBgu6E+AF4QkS+JyP+ru7V1kIjMEZGtIrJDRO5uoc0XRWSTiGwUkb/47b9ORLa7N+vy8sAT/3mCT8o+4QfTfmDLpBpjWhXsAHcSUErjGVAKPNfSAW731RJgFlAArBORFaq6ya9NNnAPcLaqHhKRVHd/EnAfkOe+znr32ENBvzPTql1lu/j9R7/n4qyLOTfz3FCHY4zp4oK9grs94xRTgB2qugtARJYB84BNfm1uApbUJQFV3e/unw28oqoH3WNfAeYAT7cjDtNE3TKp0RHRfG/K90IdjjGmGwj2Cu4/4PyF34iqtla7OgPY47ddAExt0mak+/xvAeHAQlV9uYVjMwLEdTNwM8CQIUPafB/G8fyO51m/bz0Lz1xIckxyqMMxxnQDwXZDveh3PxqYD+xt45hAA+JNE04EkA1cAGQC/xaRsUEei6o+BjwGkJeX1+xx09yBEwf4ef7PmTRwEvOz54c6HGNMNxFsN9Ry/20ReRp4tY3DCoDBftuZNE8wBcA7qloNfCIiW3GSRwFOAvE/dk0wsZrWPbTuIU7UnGDBmQsIk2DnNxhjerv2fltkA231+6wDskVkqIhEAVcBK5q0eR6n1hQikozTLbULZ/2Mi9zlWxOBi9x95hS8Wfgm//zkn9w07iaGxVsNSGNM8IIdsyincTdQMc4aFy1S1Rq3NMhKnPGIJ1V1o4gsBvJVdQUNSWETUAvcqaql7mvej5NwABbXDXab9qlbJnVo/FBuHHdjqMMxxnQzwXZDxbXnyVX1JeClJvsW+N1X4NvuremxTwJPtud1TXO/+/B3FB4t5I9z/khUuK2Ia4w5OUF1Q4nIfBGJ99tOEJHLvAvLdKQtB7fwp01/4gvZX2DSwEmhDscY0w0FO2Zxn6qW1W2o6mGci+ZMF1frq2Xh2wtJ6JPAHZPuCHU4xphuKthkEahdsNNuTQg9veVpNpZu5O4pdxPfJ77tA4wxJoBgk0W+iPxCRIaLyDAReRhY72Vg5tQVHS3ikQ8e4ZyMc5idNTvU4RhjurFgk8XtQBXwDPBX4ATwda+CMqdOVfnxuz8G4AfTfmDLpBpjTkmws6GOAQGrxpqu6ZVPX+H1gtf5bt53yejXrFKKMcaclGBnQ70iIgl+24kiYhfJdVFHqo7wwHsPkJuUy7W514Y6HGNMDxDsIHWyOwMKAP9y4qbr+dX6X1FaUcr/XPg/RITZPARjzKkLdszCJyL15T1EJIsAhf1M6H2w/wP+uu2vXJt7LWMGjAl1OMaYHiLYPzu/D7wpIq+72+fhlgY3XUd1bTWL3l5Eemw6t51+W6jDMcb0IMEOcL8sInk4CWID8ALOjCjThfxh4x/YWbaTJRcusWVSjTEdKthCgl8BvolTKnwDMA1YS+NlVk0I7S7bzaMfPspFp13EeZnnhTocY0wPE+yYxTeBycCnqjodmAiUeBaVOSmqyv3v3E+f8D7cPcVmOBtjOl6wyaJCVSsARKSPqm4BRnkXljkZL+x8gfeK3+Nbk75FSt+UUIdjjOmBgh3gLnCvs3geeEVEDtH2sqqmExysOMjP8n/GxNSJXD7y8lCHY4zpoYId4K5brHmhiKwG4oGXPYvKBO2n637Ksepj3HfmfbZMqjHGMyd9xZaqvt52K9MZ3t77Ni/uepFbxt/C8IThoQ7HGNODefqnqIjMEZGtIrJDRJqNvIrI9SJSIiIb3NtX/B6r9dvfdO0CPP17AAAbkUlEQVTuXu9EzQnuX3s/Wf2zuGn8TaEOxxjTw3lWC0JEwoElwCygAFgnIitUdVOTps+oaqAryE6o6ulexdfdPfrhoxQcLeDJ2U/SJ7xPqMMxxvRwXp5ZTAF2qOouVa0ClgHzPHy9XmPrwa38ceMfmT9iPpPTJoc6HGNML+BlssgA9vhtF7j7mvqCiHwkIs+KyGC//dEiki8i77S03reI3Oy2yS8p6R2XfdT6alm8djHxfeL5Tt53Qh2OMaaX8DJZBFptp2nxwb8DWao6HngVeMrvsSGqmgdcA/xSRJqN4KrqY6qap6p5KSm94/qCZ7Y+w0cHPuJ7k79ny6QaYzqNl8miAPA/U8ikybUZqlqqqpXu5u+BSX6P7XV/7gLW4Fw13qsVHyvmV+//irMGncUlQy8JdTjGmF7Ey2SxDsgWkaEiEgVcBTSa1SQi6X6bc4HN7v5EEenj3k8GzgaaDoz3Oj959yf41GfLpBpjOp1ns6FUtUZEbgNWAuHAk6q6UUQWA/mqugL4hojMBWqAg8D17uG5wKMi4sNJaA8EmEXVq7z26Wus2rOKOybdweC4wW0fYIwxHUhUe8YaRnl5eZqfnx/qMDxxtOoo856fR2J0Ik9f+jSRYZGhDskY00OIyHp3fLhVtuZmN/Cr939FyYkSfjn9l5YojDEhYcWEurgPSz7kma3PcE3uNYxLGRfqcIwxvZQliy6s2lfNwrcXkto3ldsn3h7qcIwxvZh1Q3VhT218ih2Hd/DI9EeIjYwNdTjGmF7Mziy6qD1H9vC7D3/HzCEzmT5keqjDMcb0cpYsuiBVZfE7i4kMi7RlUo0xXYIliy7oxV0v8k7RO3zrjG8xMHZgqMMxxhhLFl3NoYpD/HTdT5mQMoErRl0R6nCMMQawZNHl/Cz/Z5RXldsyqcaYLsW+jbqQd4veZcXOFfz32P8mOzE71OEYY0w9SxZdREVNBYvXLmZI3BBuHn9zqMMxxphG7DqLLuKxjx7js/LP+P1Fvyc6IjrU4RhjTCN2ZtEFbD+0nT/85w/MHT6XaenTQh2OMcY0Y8kixHzqY9HaRfSL6sd3874b6nCMMSYgSxYh9n9b/48PSz7ke5O/R2J0YqjDMcaYgCxZhND+4/v55fu/ZGr6VC4ddmmowzHGmBZZsgihB957gGpfNQumLbBlUo0xXZqnyUJE5ojIVhHZISLNihyJyPUiUiIiG9zbV/weu05Etru367yMMxRWf7aaVz59hVsn3MqQ/kNCHY4xxrTKs6mzIhIOLAFmAQXAOhFZEWAt7WdU9bYmxyYB9wF5gALr3WMPeRVvZzpWfYwfv/tjRiSM4LoxPS4PGmN6IC/PLKYAO1R1l6pWAcuAeUEeOxt4RVUPugniFWCOR3F2uv/54H/Yf3w/C89aaMukGmO6BS+TRQawx2+7wN3X1BdE5CMReVZEBp/ksd3OxyUf85fNf+HKUVcyIWVCqMMxxpigeJksAo3YapPtvwNZqjoeeBV46iSORURuFpF8EckvKSk5pWA7Q7WvmkVrF5ESk8I3z/hmqMMxxpigeVnuowAY7LedCez1b6CqpX6bvwce9Dv2gibHrmn6Aqr6GPAYQF5eXrNk0tX876b/Zeuhrfzygl/SL6pfqMMxhurqagoKCqioqAh1KMZj0dHRZGZmEhnZvq5vL5PFOiBbRIYChcBVwDX+DUQkXVWL3M25wGb3/krg/xORuqvULgLu8TBWzxWUF/CbDb9h+uDpXHjahaEOxxgACgoKiIuLIysry6Zv92CqSmlpKQUFBQwdOrRdz+FZslDVGhG5DeeLPxx4UlU3ishiIF9VVwDfEJG5QA1wELjePfagiNyPk3AAFqvqQa9i9Zqq8qN3fkSYhHHv1HtDHY4x9SoqKixR9AIiwoABAziV7npPq86q6kvAS032LfC7fw8tnDGo6pPAk17G11le+uQl3tr7FvdMuYe02LRQh2NMI5YoeodT/Xe2K7g9VlZZxkPrHmJ88niuHHVlqMMxpkvp1+/Uxu4uv/xydu3a1UHRdLyqqirOO+88ampq2v0cc+bMISEhgUsvbbkkUGVlJVdeeSUjRoxg6tSp7N69u92v1xJLFh77ef7POVJ5hAVnLiA8LDzU4RjTY2zcuJHa2lqGDRsW6lBaFBUVxYUXXsgzzzzT7ue48847+fOf/9xqmyeeeILExER27NjBHXfcwV133dXu12uJJQsPrStex992/I0vj/kyo5JGhTocY7osVeXOO+9k7NixjBs3rv7L1efz8bWvfY0xY8Zw6aWXcskll/Dss88CsHTpUubNa7jO94knnmDkyJFccMEF3HTTTdx2m1MY4u9//ztTp05l4sSJzJw5k3379gGwcOFCrrvuOi666CKysrJ47rnn+N73vse4ceOYM2cO1dXVAGRlZXHvvfdy5plnkpeXx/vvv8/s2bMZPnw4v/vd7wA4evQoF154IWeccQbjxo3jhRdeqI/rsssuY+nSpe3+bC688ELi4uJabfPCCy9w3XVONYjLL7+c1157DdWOnSBqK+V5pLK2ksVrF5PZL5NbJ9wa6nCMadOiv29k094jHfqcowf1577Pj2mz3XPPPceGDRv48MMPOXDgAJMnT+a8887jrbfeYvfu3Xz88cfs37+f3NxcbrjhBgDeeustrr76agD27t3L/fffz/vvv09cXBwzZsxgwgTnotdzzjmHd955BxHh8ccf56GHHuLnP/85ADt37mT16tVs2rSJM888k+XLl/PQQw8xf/58/vGPf3DZZZcBMHjwYNauXcsdd9zB9ddfz1tvvUVFRQVjxozh1ltvJTo6mr/97W/079+fAwcOMG3aNObOnYuIMHbsWNatc+bqlJeXc+655wb8DP7yl78wevTodn3OhYWFDB7sXKkQERFBfHw8paWlJCcnt+v5ArFk4ZHHP36c3Ud28+isR4mJiAl1OMZ0aW+++SZXX3014eHhDBw4kPPPP59169bx5ptvcsUVVxAWFkZaWhrTp0+vP6aoqIiUlBQA3nvvPc4//3ySkpIAuOKKK9i2bRvgTA++8sorKSoqoqqqqtHU0YsvvpjIyEjGjRtHbW0tc+Y4VYXGjRvXqN9/7ty59fuPHj1KXFwccXFxREdHc/jwYWJjY7n33nt54403CAsLo7CwkH379pGWlkZ4eDhRUVGUl5cTFxfHhg0bOvzzC3QW0dETFyxZeGDn4Z08/vHjXDrsUs4adFaowzEmKMGcAXilpS6T1rpSYmJi6i8mbK3d7bffzre//W3mzp3LmjVrWLhwYf1jffr0ASAsLIzIyMj6L9iwsLBGg9L+7eru+7dbunQpJSUlrF+/nsjISLKyshpd6FhZWUl0dHSbZxbl5eXccsstACxevLg+SbUlMzOTPXv2kJmZSU1NDWVlZfWJs6PYmEUH86mPxWsXExsZy52T7wx1OMZ0C+eddx7PPPMMtbW1lJSU8MYbbzBlyhTOOeccli9fjs/nY9++faxZs6b+mNzcXHbs2AHAlClTeP311zl06BA1NTUsX768vl1ZWRkZGU5puaeeegovlJWVkZqaSmRkJKtXr+bTTz+tf6y0tJSUlBQiIyPrzywC3UaPHs3UqVPrt4NNFOCc+dS9t2effZYZM2bYmUVXt3z7ct7f/z73n30/SdEdm9mN6anmz5/P2rVrmTBhAiLCQw89RFpaGl/4whd47bXXGDt2LCNHjmTq1KnEx8cD8LnPfY41a9Ywc+ZMMjIyuPfee5k6dSqDBg1i9OjR9e0WLlzIFVdcQUZGBtOmTeOTTz7p8PivvfZaPv/5z5OXl8fpp59OTk5O/WOrV6/mkksuafdzn3vuuWzZsoWjR4+SmZnJE088wezZs1mwYAF5eXnMnTuXG2+8kS996UuMGDGCpKQkli1b1hFvqzFV7RG3SZMmaajtP7Zfz1x6pt7w8g3q8/lCHY4xbdq0aVOoQ2hTeXm5qqoeOHBAhw0bpkVFRaqqevz4cZ06darW1NQ0alddXa2XXnqpPvfcc6EJuIn58+frli1bQh2Gqgb+98apqNHmd6ydWXSgB9c9SGVtJT+c9kO7KtaYDnLppZdy+PBhqqqq+OEPf0hamlMFISYmhkWLFlFYWMiQIUNYuHAhr776KhUVFVx00UX1M5lCqaqqissuu4xRo7r/1HlLFh3kjYI3WLl7JbedfhtZ8VmhDseYHsN/nKKp2bNn19//2c9+1gnRnJyoqCi+/OUvhzqMDmED3B3gePVxfvTOjxgeP5wbxt4Q6nCMMabD2ZlFB/j1hl9TdKyIP138JyLDbZlUY0zPY2cWp2hj6UaWbl7KF0d+kYmpE0MdjjHGeMKSxSmo8dWw6O1FJEUn8c1JtkyqMabnsmRxCpZuXsrmg5u5Z8o99I/qH+pwjOl2rER5c5988glTp04lOzubK6+8kqqqqoDtfvKTnzBixAhGjRrFypUrOyrkFlmyaKfCo4Us2bCECzIvYNZps0IdjjG9Tk8tUX7XXXdxxx13sH37dhITE3niiSeatdm0aRPLli1j48aNvPzyy3zta1+jtra2I0NvxtNkISJzRGSriOwQkbtbaXe5iKiI5LnbWSJyQkQ2uLffeRnnyVJ3mVSAe6fea9dUGHOK1EqU138Oq1at4vLLLwfguuuu4/nnn2/W7oUXXuCqq66iT58+DB06lBEjRvDee++d1Gd+sjybDSUi4cASYBZQAKwTkRWquqlJuzjgG8C7TZ5ip6qe7lV8p2Ll7pW8Wfgmd02+i/R+6aEOx5iO8c+7ofjjjn3OtHFw8QNtNrMS5U4hwdTUVBISEoiIcL6aMzMzKSwsbNa2sLCQadOm1W+31K4jeTl1dgqwQ1V3AYjIMmAesKlJu/uBh4DvehhLhymrLOOB9x5gzIAxXJ1zdajDMaZHsBLljpKSkmb7AvVcaCeUJG/Ky2SRAezx2y4Apvo3EJGJwGBVfVFEmiaLoSLyAXAE+IGq/tvDWIP28PqHOVx5mN/O/K0tk2p6liDOALwS6Muvtf3QM0uU5+bmcvjwYWpqaoiIiKCgoIBBgwY1a1tXkrxOS+06kpdjFoHSXP2/qIiEAQ8D3wnQrggYoqoTgW8DfxGRZtONRORmEckXkfxAGbmjrd+3nuXbl/Ol0V8id0Cu569nTG9hJcqdEuUiwvTp0+vHZZ566qlG4zJ15s6dy7Jly6isrOSTTz5h+/btTJkyxZP3VsfLZFEADPbbzgT2+m3HAWOBNSKyG5gGrBCRPFWtVNVSAFVdD+wERjZ9AVV9TFXzVDWv7nTUK1W1VSxau4iMfhl8dcJXPX0tY3qb+fPnM378eCZMmMCMGTMalSjPzMxk7Nix3HLLLQFLlAONSpTPnDkzYInyc889t0OXGfV37bXXkp+fT15eHkuXLj2lEuUPPvggv/jFLxgxYgSlpaXceOONAKxYsYIFCxYAMGbMGL74xS8yevRo5syZw5IlSwgP97inI5jStO254XRx7QKGAlHAh8CYVtqvAfLc+ylAuHt/GFAIJLX2el6XKP/Nht/o2D+O1X8X/NvT1zGmM1mJcu9ZifK2k1CNiNwGrATCgSdVdaOILHaDW9HK4ecBi0WkBqgFblXVg17F2pZdZbv4/Ue/5+KhF3NOxjmhCsOYXslKlHcNoq0MDHUneXl5mp+f3+HP61MfN6y8gW2HtrHishUkx3hzGmtMKGzevJncXBt/6y0C/XuLyHpVzWvrWLuCuw3P73ie9fvW892871qiMMb0WpYsWnHgxAF+lv8zJg2cxPwR80MdjjHGhIwli1Y8tO4hKmoqWHDmAivpYYzp1SxZtODNwjf55yf/5KZxNzEsvusWKjPGmM5gySKAumVSh8YP5cZxN4Y6HGN6LCtR3tyvf/1rRowYgYhw4MCBFts99dRTZGdnk52d7dnFhv4sWQTw2w9/S+HRQu478z6iwqNCHY4xJoCeWqL87LPP5tVXX+W0005rsc3BgwdZtGgR7777Lu+99x6LFi3i0KFDHRFyiyxZNLG5dDN/3vRnvpD9BSYNnBTqcIzpFdRKlNebOHEiWVlZrbZZuXIls2bNIikpicTERGbNmsXLL78c9Gu0h5eFBLudWl8ti9YuIqFPAndMuiPU4RjTqR5870G2HNzSoc+Zk5TDXVPuarOdlSh3CgmOHj06qM+1sLCQwYMbqil19xLl3c7TW55mY+lGfnreT4nvEx/qcIzpNaxE+ckJdDF1dy5R3q0UHS3ikQ8e4dyMc5mdNTvU4RjT6YI5A/BKS5UkWqsw0RNLlAd7ZpGZmdmoAm9BQQEXXHBBUMe2l41Z4PxH+/G7Pwbg+9O+b9dUGNPJrET5hqATBcDs2bP517/+xaFDhzh06BD/+te/mD3b2z9yLVkAr3z6Cq8XvM7XT/86Gf0yQh2OMb2OlShv8Mgjj5CZmUlBQQHjx4/nK1/5CgD5+fn195OSkvjhD3/I5MmTmTx5MgsWLKjvgvNMMKVpu8OtvSXKyyrLdPoz0/WKFVdodW11u57DmO7KSpR7z0qU9xBVtVWMTR7LLRNuISKs138cxnQ5VqK8a7AS5cb0YlaivHexEuXGGGM8ZcnCmF6up/QumNad6r+zp8lCROaIyFYR2SEid7fS7nIRURHJ89t3j3vcVhGxCx+M8UB0dDSlpaWWMHo4VaW0tJTo6Oh2P4dnI7oiEg4sAWYBBcA6EVmhqpuatIsDvgG867dvNHAVMAYYBLwqIiNVtdareI3pjeqmaJaUlIQ6FOOx6OhoMjMz2328l9N/pgA7VHUXgIgsA+YBm5q0ux94CPiu3755wDJVrQQ+EZEd7vOt9TBeY3qdyMjIRuUvjGmJl91QGcAev+0Cd189EZkIDFbVF0/2WGOMMZ3Hy2QRqGZGfceoiIQBDwPfOdlj/Z7jZhHJF5F8O402xhjveJksCoDBftuZwF6/7ThgLLBGRHYD04AV7iB3W8cCoKqPqWqequbVVZ80xhjT8Ty7KE9EIoBtwIVAIbAOuEZVN7bQfg3wXVXNF5ExwF9wxikGAa8B2a0NcItICfBpS48HIRloeQ3D0LG4To7FdXIsrpPTE+M6TVXb/GvbswFuVa0RkduAlUA48KSqbhSRxTi1SFa0cuxGEfkrzmB4DfD1tmZCBfNmWyMi+cFcxdjZLK6TY3GdHIvr5PTmuDwthqSqLwEvNdm3oIW2FzTZ/jHwY8+CM8YYEzS7gtsYY0ybLFk0eCzUAbTA4jo5FtfJsbhOTq+Nq8dUnTXGGOMdO7MwxhjTpl6bLETkChHZKCI+/wKGAdoFVQyxA+NKEpFXRGS7+zOxhXa1IrLBvbU4s6wD4mn1/YtIHxF5xn38XRHJ8iqWk4jpehEp8ft8vuJ1TO7rPiki+0XkPy08LiLyiBv3RyJyRheJ6wIRKfP7vAJOQvEgrsEislpENru/i98M0KbTP7Mg4+r0z0xEokXkPRH50I1rUYA23v0+BrOcXk+8AbnAKGANkNdCm3BgJzAMiAI+BEZ7HNdDwN3u/buBB1tod7QTPqM23z/wNeB37v2rgGe6QEzXA78Owf+p84AzgP+08PglwD9xKhRMA97tInFdALwYgs8rHTjDvR+Hc11W03/LTv/Mgoyr0z8z9zPo596PxCm+Oq1JG89+H3vtmYWqblbVrW00qy+GqKpVQF0xRC/NA55y7z8FhHJtyGDev3+8zwIXikigci2dGVNIqOobwMFWmswD/qSOd4AEEUnvAnGFhKoWqer77v1yYDPNa8B1+mcWZFydzv0Mjrqbke6t6aCzZ7+PvTZZBCkUBQ0HqmoROP9pgdQW2kW7dbHeERGvEkow77++jarWAGXAAI/iCTYmgC+43RbPisjgAI+HQlcukHmm273xT7eCQqdyu0sm4rdUgSukn1krcUEIPjMRCReRDcB+4BVVbfHz6ujfR08vygs1EXkVSAvw0PdV9YVgniLAvlOePtZaXCfxNENUda+IDANWicjHqrrzVGNrIpj378ln1IpgXu/vwNOqWikit+L8pTXDw5iC1dmfVbDexyn5cFRELgGeB7I768VFpB+wHPiWqh5p+nCAQzrlM2sjrpB8ZupUsjhdRBKAv4nIWFX1H4vy7PPq0clCVWee4lMEVdDwZLUWl4jsE5F0VS1yT7f3t/Ace92fu8SpqzURpy+/IwXz/uvaFIhTDyweb7s82oxJVUv9Nn8PPOhhPCfDk/9Pp8r/i1BVXxKR34hIsqp6XgNJRCJxvpCXqupzAZqE5DNrK65Qfmbuax52f+/nAP7JwrPfR+uGat06IFtEhopIFM6AkWczj1wrgOvc+9cBzc6ARCRRRPq495OBs2m+qFRHCOb9+8d7ObBK3dE1j7QZU5M+7bk4fc5dwQrgy+4Mn2lAWV2XYyiJSFpdv7aITMH5Xiht/agOeV0BngA2q+ovWmjW6Z9ZMHGF4jMTkRT3jAIRiQFmAluaNPPu97EzR/O70g2Yj5OFK4F9wEp3/yDgJb92l+DMhtiJ033ldVwDcKrsbnd/Jrn784DH3ftnAR/jzAT6GLjRw3iavX9gMTDXvR8N/B+wA3gPGNYJn1FbMf0E2Oh+PquBnE76P/U0UARUu/+3bgRuBW51HxecpYZ3uv9uAWfhhSCu2/w+r3eAszoprnNwukg+Aja4t0tC/ZkFGVenf2bAeOADN67/AAvc/Z3y+2hXcBtjjGmTdUMZY4xpkyULY4wxbbJkYYwxpk2WLIwxxrTJkoUxxpg2WbIw5iSIyNG2W7V6/LPuVfeISD8ReVREdrpVRN8QkakiEuXe79EXzZruxZKFMZ3ErR8Urqq73F2P41xdm62qY3Cq5SarUyDxNeDKkARqTACWLIxpB/eK4p+KyH9E5GMRudLdH+aWftgoIi+KyEsicrl72LW4V+SLyHBgKvADVfWBU7pFVf/htn3ebW9Ml2Cnuca0z/8DTgcmAMnAOhF5A6f0ShYwDqdi8GbgSfeYs3GupgYYA2xQpzBcIP8BJnsSuTHtYGcWxrTPOTiVbWtVdR/wOs6X+znA/6mqT1WLccqN1EkHSoJ5cjeJVIlIXAfHbUy7WLIwpn1aWlCmtYVmTuDU7gGnrtAEEWntd7APUNGO2IzpcJYsjGmfN4Ar3cVoUnCWLn0PeBNn4aUwERmIs/xmnc3ACAB11h7JBxb5VS/NFpF57v0BQImqVnfWGzKmNZYsjGmfv+FU//wQWAV8z+12Wo5T2fU/wKM4K6yVucf8g8bJ4ys4i2DtEJGPcdbeqFurYTrwkrdvwZjgWdVZYzqYiPRTZwW1AThnG2erarG7BsFqd7ulge2653gOuEfbXifemE5hs6GM6XgvuovURAH3u2ccqOoJEbkPZ53kz1o62F3U6XlLFKYrsTMLY4wxbbIxC2OMMW2yZGGMMaZNliyMMca0yZKFMcaYNlmyMMYY0yZLFsYYY9r0/wMeg33ISVMlnQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x_axis=np.log10(C_s)\n",
    "for j,gamma in enumerate(gamma_s):\n",
    "    plt.plot(x_axis,np.array(accuracy_s[:,j]),label='log(gamma)='+str(np.log10(gamma)))\n",
    "plt.legend()\n",
    "plt.xlabel('log(C)')\n",
    "plt.ylabel('accuracy')\n",
    "plt.savefig('RBF_SVM_PCA_Otto.png')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/liu/anaconda3/lib/python3.7/site-packages/sklearn/model_selection/_split.py:2026: FutureWarning: From version 0.21, test_size will always complement train_size unless both are specified.\n",
      "  FutureWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C=0.1 and gamma=0.1:,accuracy_train=0.70325，accuracy_val=0.707\n",
      "C=0.1 and gamma=1.0:,accuracy_train=0.737125，accuracy_val=0.739\n",
      "C=0.1 and gamma=10.0:,accuracy_train=0.451875，accuracy_val=0.4155\n",
      "C=1.0 and gamma=0.1:,accuracy_train=0.747625，accuracy_val=0.7495\n",
      "C=1.0 and gamma=1.0:,accuracy_train=0.832875，accuracy_val=0.7865\n",
      "C=1.0 and gamma=10.0:,accuracy_train=0.975625，accuracy_val=0.6455\n",
      "C=10.0 and gamma=0.1:,accuracy_train=0.79175，accuracy_val=0.7755\n",
      "C=10.0 and gamma=1.0:,accuracy_train=0.950625，accuracy_val=0.7885\n",
      "C=10.0 and gamma=10.0:,accuracy_train=0.999，accuracy_val=0.6625\n",
      "C=100.0 and gamma=0.1:,accuracy_train=0.87675，accuracy_val=0.7925\n",
      "C=100.0 and gamma=1.0:,accuracy_train=0.992125，accuracy_val=0.765\n",
      "C=100.0 and gamma=10.0:,accuracy_train=1.0，accuracy_val=0.66\n",
      "C=1000.0 and gamma=0.1:,accuracy_train=0.9495，accuracy_val=0.7745\n",
      "C=1000.0 and gamma=1.0:,accuracy_train=0.99925，accuracy_val=0.7615\n",
      "C=1000.0 and gamma=10.0:,accuracy_train=1.0，accuracy_val=0.66\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl8VPX1+P/XyUZCCFlIQkIChiWQsIqExV0QBK0F+Vbr9mn1o3Vpq21ta13aImD7q9rF1k9pq1Vb2w8V+xGr1FqpCmhVVIKiln0RJSGBECCEJeuc3x/3Jpkkk2QIuZks5/l4zCNz77zvzJmBzMl9v9/3vEVVMcYYY1oTFuoAjDHGdH2WLIwxxrTJkoUxxpg2WbIwxhjTJksWxhhj2mTJwhhjTJssWRhjjGmTJQtjjDFtsmRhjDGmTRGhDqCjJCcna1ZWVqjDMMaYbmX9+vUHVDWlrXY9JllkZWWRn58f6jCMMaZbEZFPg2ln3VDGGGPa5GmyEJE5IrJVRHaIyN0BHh8iIqtF5AMR+UhELvF77B73uK0iMtvLOI0xxrTOs24oEQkHlgCzgAJgnYisUNVNfs1+APxVVX8rIqOBl4As9/5VwBhgEPCqiIxU1Vqv4jXGGNMyL88spgA7VHWXqlYBy4B5Tdoo0N+9Hw/sde/PA5apaqWqfgLscJ/PGGNMCHiZLDKAPX7bBe4+fwuB/xKRApyzittP4lhE5GYRyReR/JKSko6K2xhjTBNeJgsJsK/pSktXA39U1UzgEuDPIhIW5LGo6mOqmqeqeSkpbc78MsYY005eTp0tAAb7bWfS0M1U50ZgDoCqrhWRaCA5yGONMcZ0Ei+TxTogW0SGAoU4A9bXNGnzGXAh8EcRyQWigRJgBfAXEfkFzgB3NvCeh7EaY0Kk1qfsL69g7+ETFB52fkaECZmJMWQm9iUzMYb4mEhEAnU4mM7iWbJQ1RoRuQ1YCYQDT6rqRhFZDOSr6grgO8DvReQOnG6m69VZFHyjiPwV2ATUAF+3mVDGdE/lFdXsPVyXDE6wt/5WQeHhExQfqaDW16yXuZHYqPD6xJGZGEOGXyLJTOxLYl9LJl4T57u5+8vLy1O7gtu0yecDXw1orfPTV+vcGm3XgPoabwfap7UNxzd6zgCv0S8VUnIgcSiE95jCCdTU+thXXlmfAAr9EkHddnlFTaNjIsKE9IRo0uNjyEiIYVBCNIMSYhiU4Gynx0fj88GeQ8cpOHSCAvdn4eETzvbB45RXNn7OmMjw+kSSmdjXTSYNCWVAbJQlkxaIyHpVzWurXc/5X2s6R9Vx2P1vOF7a5Mu06RduEF/AWtv4+Ba/gAN9IQfzJR3gS7v5PInOFRYJA0ZAykgneSSPhJRRMCAbIqNDG1sTqsqRihq/M4GGbqK6W/GRCpqeFCT2jWRQQgyDk/oybdiAZskguV8fwsPa/uKO7xvP2Iz4gI+Vnaim0C+ROMnEuf/+Z4cpO1HdqH10ZBgZCQ3Jo/GZSQwp/fpYMmmDJQvTtsqjsP1fsOkF52f18eCPlTCQcAiLgLBw59ZoOyLAvhbaRPRp2JbwhrZhEYH3BdyOgLCwJtuB9rUQh4Q12Q5vHGPA1w2DI3uhZCsc2Or8LP4PbP67k0DrPqeE05wEUp9IRjn3+8R58s9aVeNj35GKRl1DhYcrKCprODs42uQv+KjwMNITohkUH8OZw5PJ8EsEg9yzhL5R3n+txMdEEh8TyehB/QM+Xl5R7ZyJHGx+ZvJRwWEOHW+cTKIiwshMaJ5E6s5OUvr1ISyIBNeTWTeUCayiDLatdBLEjlehpgJiUyH3884taWgbX9J1SaB3/4K1qroCDu50kod/IindAbVVDe36Z7hnIE0SSeyAFp9aVTl8vLrxGEFZ48Swv7ySpr/+A2Kj6r/0684E/BNBcmzP+NI8VlnjJg+/MxO/M5XSY1WN2keFhzEoIbrFcZPUuOigzpa6omC7oSxZmAYnDsHWfzoJYucq5wsrbpCTHEbPgyHTnARgvFVbA4d2u8ljC5Rsc+9vg+pjDc1iBnA0bjgHYoZSED6Y7ZrBRxUD2Vjel71llZyobjwnJCoirGGMID6mSTJwkkN0pP37Ahyvcrrf9riJpODQcTeZOLcDRysbtY8MF9LjG5+NON1eMWQm9WVgXB8iwrtm3VZLFiY4x0phy4tOgvjkdadvP36wkxxGz4OMPKeLxnQaVeXgsar62UJ1ZwJFh49RcbCA2LIdpFTsZoQUMiKskGwpJEEaksiJsFhKY7I43n84vuRR9EnPJX7IWBLThyM9aHA9lCqqaxsG3Ou6ufzu7y9vnEwiwoS0+OhGg+7+Yyjp8dEhSyY2wG1adnS/01++6QXY/aYzAJyYBWfeBqPnwqAzrPvIQxXVtRSVBZ5KWrevssbX6JjoyDD3TGAQyYOG0z8hhoiEaKoTYjgcH010RDnRh7dDyVZiSraSeWArlLwLRSvgY/dJIqIhOdsdC3G7tJJHQdIwiIjq/A+iG4uODGd4Sj+Gp/QL+Hjdv3GB34yuujOTN7cfYF95RaMuwPAwIa1/dONZXAkN99MTookM8ZmJnVn0Fkf2NiSIT98G1JmBU3cGkTbOEkQHqqypZcf+o2wpKmfrvnI+Kz3OXnfg+MDRxv3hIpAa16fRjKFB8Y0Hjtt9HcGJQ3Bgu9ud5Tc2cvizhjZhEU7CSBnVOJEMyIaovqf4SZhAqmp8FJU1PjPxPztpOsssTPBLJn7jJgnumUlCNH0i2teFaN1QxvlC2LTCSRAF7gXwqaMbEkRKjiWIU6SqFJVVsKX4CJuLytlaXM6W4iPsLDlWf6FZVEQYpyX19UsGjaeSDuwfTVREJ//VWHXMTSJ+A+slW+HgLudMEwCBhCFOEmmaSKIDT2k1HaOqxkdxWQUFhxsSif/ZSVHZiUbJZMyg/vzjG+e267WsG6q3OrirIUHsfd/ZlzYeZvwAcuc5v+imXY5V1rBtXzlbisvZUnSEze7PI34XnWUkxJCbHsdFo9PISY8jJ60/WQP6dr3BzahYGHS6c/NXUxV4htau16HWrx8+Lr2FGVrJ9gdIB4iKCGPIgL4MGRD4zK661kkmdeMmnfHHhp1Z9AQHtsOm550EUex2UA86wz2DmOt0MZig+XzKZweP158tbCk+wpbicj4tbbi+JDYqnJz0/uSkxdX/HJUWR//oyBBG7iFfrTtDa1vDDK2SLc521dGGdjGJfhcb+iWS/hmWRLoo64bqyVRh/2YnOWxeAfvdxQcHT4XcuU6CSBgS2hi7icPHq+rPFLYUO2cNW4vL66edhglkJceSm9afUWlx5KTFkZven4yEmB5xvcEpU3UvONzSPJGcONjQLqpfw9Xq9V1ao5yJFTYdO6QsWfQ0qs5Zw6YXnFvpdkDgtLOcM4jcz0P/QaGOssuqrvWxq+RY/VlCXXIoKquob5PYN5KctP7kpMeR6/7MTo0jJsq+zNrl2IHmA+sl26Dcb7WB8D5u+ZNRjRNJv4HODK3wKKdEik3f9oyNWfQEqs64Q12COLTbKQuRdS5M+yrkXApxA0MdZZeiqpSUV7pnCUfYUlTO5uJyduwvp7rW+cMoMlwYntKPacMGNOpGSo2z+kAdKjYZYs+BrHMa768oaz5Da+/7sPFvtFi7KyzCSRzhkU6Cqb/vJpS6xOK/L2Bb92dEnwBto9z2AfbXP38rbXt4xQJLFl2NzwcF65zupU0vQNke5xdl6Plwzrch53POL6GhorqW7fuOstlNCnVnDQf9SjWk9Y8mJz2O80emuIkhjmHJ/Tp/9pFpEB0PmXnOzV/VcafUSclWpwurtsq51VQ13K+t9rsfYF9NlVPLrFG7amdw3r+tryZwbKdEAiQi/8QTKOFEuu2DTHQttY1Jaj5ZoYNZsugKfLXw2TsNYxDlRc5/guEzYPq9MOpiZ+Cwl1JVCg6daNR9tLn4CLsPHKufPhgdGcaotP7Myh1YPwspJy2OxFi72KzbiOoL6eOdm9d8PvBVQ01lgMRS5SaXAPtbbN9K8mq2v9qZFNDs+KbxVLb9Pupk5MFNr3n3eWHJInRqa+DTN90E8SIc2+9cYTtipjMGMXJ2r5zLXl5RzdZip+toq3vGsLW4vNH6BUOS+pKTFsel4weR63YjDUnq220LuZkQCAuDsD7OX+pdlarzh2Srycvd7oTy9pYsOlNttVN/qS5BnDgIkX0h+yInQWRfBH0Clw/oaWp9yu7SY/XdR3VTVAsOnahvExcdQW5af+afkVE/8DxyYBz9+th/W9MLiDgLZYVHAKG/kt7T3zoRmQP8CmdZ1cdV9YEmjz8MTHc3+wKpqprgPlZLQ1Wbz1R1rpexeqamEnaudhLE1n84g3tRcTBqjpMghl/Y40sqHDxW1egiti3F5WzbV15f/yg8TBiWHMvEIYlcPWVI/aDzoPhoG3A2povwLFmISDiwBJgFFADrRGSFqm6qa6Oqd/i1vx2Y6PcUJ1TV2xEbr1SfcNaA2LQCtr0MlUecLqVRn3OugRg2vcutitYRKmtq2bm/YXrq5qIjbC0ub1SBM7lfH3LT4/jStNPqZyGNSO1npbGN6eK8PLOYAuxQ1V0AIrIMmAdsaqH91cB9HsbjrcqjsOMV5wxi27+cdQdiktyrqC+Doef1mMqeqkrxkQp3WqqTELYUlbOz5Cg1fvWQslP7cW52CrnugPOotDhS4rpwH7ExpkVeJosMYI/fdgEwNVBDETkNGAqs8tsdLSL5QA3wgKo+71Wg7VZxxF1N7nm/1eRSYMKVTpI47Ry3v7H7KztRzb+3l7Bqy37e2FbSqHJqRkIMOWlxzBydSk5af3LT48gaENv16iEZY9rNy2+yQJ3NLV0ufhXwrKr6L+01RFX3isgwYJWIfKyqOxu9gMjNwM0AQ4Z0UnmL+tXkVsDO19zV5NLhjC+7q8md2SPKF6gqO/YfZdWW/azasp/1nx6ixqck9I3k/JEp5J2WyCj3bCE+pofWQzLG1PMyWRQAg/22M4G9LbS9Cvi6/w5V3ev+3CUia3DGM3Y2afMY8Bg45T46JOpAjpU6g9ObXoBda5wLevpnwuSbnASROblHlCOoqK5l7a5SVrsJom5mUk5aHDefN4wZOamcPjjBzhiM6YW8TBbrgGwRGQoU4iSEa5o2EpFRQCKw1m9fInBcVStFJBk4G3jIw1iba3E1ua87pb4zesZqcnsPn2DVlv2s3rKft3YeoKLaR0xkOGePGMBXLxjO9FGpDEqICXWYxpgQ8yxZqGqNiNwGrMSZOvukqm4UkcVAvqqucJteDSzTxhUNc4FHRcQHhOGMWbQ0MN5xjhT5rSb3Fs5qciPgnG+5q8mN7/YJoqbWxwd7DtcniC3F5QAMTorhyrzBTM9JZdqwATY7yRjTiFWdrSiDD/7XSRB73nX2peQ2rCaXmtvtE8ShY1W8vs0ZnH59WwllJ6qJCBPyshKZkZPKjJxUhqf0s2sajOmFrOpssHy18K8fwsDRMP0HznUQKaNCHdUpUVU2F5Wzeqsz9vDBZ4fwKST3i2Jm7kBm5KRy7sjknrtQjzGmw1my6JsEd2yE/umhjuSUHK+q4a0dpazasp81W/fXr9MwLiOe22ZkMyMnlfEZ8bZgjzGmXSxZQLdNFJ+VHmfVln2s2lrCO7tKqarx0a9PBOeMSOaOmalcMCqF1P4970pxY0zns2TRjVTX+li3+2D91NadJccAGJYSy5enncaMnFTyspJsrQZjTIezZNHFlZRXsmbrflZv3c+/tx2gvLKGqPAwpg5L4r+mncb0UalkJceGOkxjTA9nyaKL8fmUjwvLnKmtW/fzUUEZAAP79+HSCelMH5XK2SOSibUy3caYTmTfOF1AeUU1/95+wB2cLuHA0UpEYOLgBL570Uim56QyOr2/TW01xoSMJYsQUFV2lhyrH3tYt/sgNT6lf3QE549KZUZOCuePTCXJlgQ1xnQRliw6SUV1Le9+0jA4/dnB4wCMGhjHV8516i6dMcTqLhljuiZLFh4qLquor9r61o4DnKiupU9EGGePSOam84YxfVQKmYk9e5U8Y0zPYMmiA9X6lA17DrkJooTNRUcAZ72HyydlMiMnlTOHW90lY0z3Y8niFB0+7tRdWu3WXTp0vJrwMGHSaYncfXEOM3JSyU61ukvGmO7NksVJUlW27iuvr9q6/lOn7lJSbBTTR6UyPSeV87JTiO9rdZeMMT2HJYsgnKiq5e2dDVNbCw87iwKNGdSfr08fwfScVCZkJhBudZeMMT2UJYsW7Dl4vL5q69qdpVTW+OgbFc45I5K5fYaTIAZa3SVjTC9hycJVXetj/aeH6qe2bt9/FICsAX25ZuoQZuSkMmVoEn0ibHDaGNP79Ppksf9IBYte3MQb20oor6ghMlyYOnQAV01xEsRQq7tkjDHeJgsRmQP8CmdZ1cdV9YEmjz8MTHc3+wKpqprgPnYd8AP3sR+p6lNexNg/JpL/FJZxydh0puekck52Mv2s7pIxxjTi2bKqIhIObANmAQXAOuDqltbSFpHbgYmqeoOIJAH5QB6gwHpgkqoeaun12r2sKs4MJ5vaaozpjYJdVtXL2hJTgB2quktVq4BlwLxW2l8NPO3enw28oqoH3QTxCjDHq0AtURhjTOu8TBYZwB6/7QJ3XzMichowFFh1sscaY4zxnpfJItCf6y31eV0FPKuqtSdzrIjcLCL5IpJfUlLSzjCNMca0xctkUQAM9tvOBPa20PYqGrqggj5WVR9T1TxVzUtJSTnFcI0xxrTEy2SxDsgWkaEiEoWTEFY0bSQio4BEYK3f7pXARSKSKCKJwEXuPmOMMSHg2RxRVa0RkdtwvuTDgSdVdaOILAbyVbUucVwNLFO/aVmqelBE7sdJOACLVfWgV7EaY4xpnWdTZzvbqUydNcaY3qorTJ01xhjTQ1iyMMYY0yZLFsYYY9pkycIYY0ybLFkYY4xpkyULY4wxbbJkYYwxpk2WLIwxxrTJkoUxxpg2WbIwxhjTJksWxhhj2mTJwhhjTJssWRhjjGmTZyXKjenNVJXDlYcpOlZE8bFiqnxVREok4WHhhEs4EWER9bdwCSc8LJwIabyv6f1wCScyzHmOMLG/80znsmRhTDtU1lay79g+io4V1d+KjxVTdLThfkVthWevL0hDwpEIJ9m0kmSa7QsLbz15tZDQIsMiT+l5Wzqu6ePhYeGefXY9kSBEhUd5+hqWLIxpQlU5WHHQ+fJvIRmUVpQ2Oy45Jpn02HSyE7M5L/M80mPTSY9NJy02jeiIaGp8NdRoDTW+Gmp9tfXbdfdrNfC+al+1s123v8lzBDrOv43/4zW+GipqKprtaxZPgOcyXdf45PEs/dxST1/DkoXpdSpqKuoTQaCEUHysmMraykbHxETEkBabRnpsOqOSRtXfr7sNjB3o+V92oaSq+NQXMMk0SmjtSF6BEpVPfaF+y91Kat9Uz1/D02QhInOAX+Esq/q4qj4QoM0XgYWAAh+q6jXu/lrgY7fZZ6o618tYTc/gUx8HKw7WnwE0TQjFx4o5WNF4hV5BSIlJIa1fGjlJOUwfPL1ZMojvE4+IhOhdhZ6ION1DhPfopGha5lmyEJFwYAkwCygA1onIClXd5NcmG7gHOFtVD4mIf3o8oaqnexWf6Z6OVx+n+HgxxUcDdBG5P6t91Y2OiYmIYVDsINL6pTF6wOhG3UPpsekM7DuQyPDIEL0jY7oHL88spgA7VHUXgIgsA+YBm/za3AQsUdVDAKq638N4TBfnUx8HThxoSABHi5slg8OVhxsdEyZhpMSkkB6bztgBY5l52sxGZwRpsWn0j+rfq88KjOkIXiaLDGCP33YBMLVJm5EAIvIWTlfVQlV92X0sWkTygRrgAVV93sNYTSc4Vn2s8RjB0cZdRPuO72s2kBobGVv/xT8ueRzp/dIbdRGl9E0hMszOCozxmpfJItCfchrg9bOBC4BM4N8iMlZVDwNDVHWviAwDVonIx6q6s9ELiNwM3AwwZMiQjo7fnIRaXy0lJ0paTQZHqo40OiZcwkntm0p6bDoTUiY0nBH4JYS4qLgQvSNjjD8vk0UBMNhvOxPYG6DNO6paDXwiIltxksc6Vd0LoKq7RGQNMBFolCxU9THgMYC8vLymich45I2CN/hg/weNEsK+4/uo1dpG7eKi4uoTwOmppzdKBumx6STHJBMRZhPyjOkOvPxNXQdki8hQoBC4CrimSZvngauBP4pIMk631C4RSQSOq2qlu/9s4CEPYzVBWle8jq+/9nUiJIKBsQNJi03jjIFnNBowrrvfL6pfqMM1xnQQz5KFqtaIyG3ASpzxiCdVdaOILAbyVXWF+9hFIrIJqAXuVNVSETkLeFREfDj1qx7wn0VlQqOytpLFaxeT0S+D5+Y+R9/IvqEOyRjTSYJKFiKyHHgS+Kdq8FfLqOpLwEtN9i3wu6/At92bf5u3gXHBvo7pHI9//Di7j+zm0ZmPWqIwppcJthrZb3G6kLaLyAMikuNhTKYL2nl4J49//DifG/Y5zso4K9ThGGM6WVDJQlVfVdVrgTOA3cArIvK2iPy3iNi8xR7Opz4Wr11MbGQsd+bdGepwjDEhEHSdYxEZAFwPfAX4AKeMxxnAK55EZrqM5duX8/7+9/nOpO8wIGZAqMMxxoRAsGMWzwE5wJ+Bz6tqkfvQM+6Fc6aHKjlewsP5DzM5bTKXjbgs1OEYY0Ik2NlQv1bVVYEeUNW8DozHdDEPrnuQytpKFkxbYCUzjOnFgu2GyhWRhLoNEUkUka95FJPpIt4oeIOVu1dy8/ibyYrPCnU4xpgQCjZZ3OSW4ADALfx3kzchma7gePVxfvTOjxgeP5wbxt4Q6nCMMSEWbDdUmIiIe11EXflxK2rfgy3ZsISiY0X86eI/WfluY0zQyWIl8FcR+R1OMcBbgZdbP8R0VxtLN/K/m/+XL478IhNTJ4Y6HGNMFxBssrgLuAX4Kk412X8Bj3sVlAmdGl8Ni95eRFJ0Et+c9M1Qh2OM6SKCShZuiY/fujfTgy3dvJTNBzfz8/N/Tv+o/qEOxxjTRQR7nUU28BNgNBBdt19Vh3kUlwmBwqOFLNmwhPMzz2fWabNCHY4xpgsJdjbUH3DOKmqA6cCfcC7QMz2EqvKjd34EwPenft+uqTDGNBJssohR1dcAUdVPVXUhMMO7sExnW7l7JW8WvsntE28nvV96qMMxxnQxwQ5wV4hIGE7V2dtwFjNK9S4s05nKKst44L0HGDNgDNfkNF2fyhhjgj+z+BbQF/gGMAn4L+A6r4Iynevh9Q9zuPIw9515H+Fh4aEOxxjTBbV5ZuFegPdFVb0TOAr8t+dRmU6zft96lm9fzvVjrid3QG6owzHGdFFtnlmoai0wSWzEs8epqq1i0dpFDIodxFcnfDXU4RhjurBgu6E+AF4QkS+JyP+ru7V1kIjMEZGtIrJDRO5uoc0XRWSTiGwUkb/47b9ORLa7N+vy8sAT/3mCT8o+4QfTfmDLpBpjWhXsAHcSUErjGVAKPNfSAW731RJgFlAArBORFaq6ya9NNnAPcLaqHhKRVHd/EnAfkOe+znr32ENBvzPTql1lu/j9R7/n4qyLOTfz3FCHY4zp4oK9grs94xRTgB2qugtARJYB84BNfm1uApbUJQFV3e/unw28oqoH3WNfAeYAT7cjDtNE3TKp0RHRfG/K90IdjjGmGwj2Cu4/4PyF34iqtla7OgPY47ddAExt0mak+/xvAeHAQlV9uYVjMwLEdTNwM8CQIUPafB/G8fyO51m/bz0Lz1xIckxyqMMxxnQDwXZDveh3PxqYD+xt45hAA+JNE04EkA1cAGQC/xaRsUEei6o+BjwGkJeX1+xx09yBEwf4ef7PmTRwEvOz54c6HGNMNxFsN9Ry/20ReRp4tY3DCoDBftuZNE8wBcA7qloNfCIiW3GSRwFOAvE/dk0wsZrWPbTuIU7UnGDBmQsIk2DnNxhjerv2fltkA231+6wDskVkqIhEAVcBK5q0eR6n1hQikozTLbULZ/2Mi9zlWxOBi9x95hS8Wfgm//zkn9w07iaGxVsNSGNM8IIdsyincTdQMc4aFy1S1Rq3NMhKnPGIJ1V1o4gsBvJVdQUNSWETUAvcqaql7mvej5NwABbXDXab9qlbJnVo/FBuHHdjqMMxxnQzwXZDxbXnyVX1JeClJvsW+N1X4NvuremxTwJPtud1TXO/+/B3FB4t5I9z/khUuK2Ia4w5OUF1Q4nIfBGJ99tOEJHLvAvLdKQtB7fwp01/4gvZX2DSwEmhDscY0w0FO2Zxn6qW1W2o6mGci+ZMF1frq2Xh2wtJ6JPAHZPuCHU4xphuKthkEahdsNNuTQg9veVpNpZu5O4pdxPfJ77tA4wxJoBgk0W+iPxCRIaLyDAReRhY72Vg5tQVHS3ikQ8e4ZyMc5idNTvU4RhjurFgk8XtQBXwDPBX4ATwda+CMqdOVfnxuz8G4AfTfmDLpBpjTkmws6GOAQGrxpqu6ZVPX+H1gtf5bt53yejXrFKKMcaclGBnQ70iIgl+24kiYhfJdVFHqo7wwHsPkJuUy7W514Y6HGNMDxDsIHWyOwMKAP9y4qbr+dX6X1FaUcr/XPg/RITZPARjzKkLdszCJyL15T1EJIsAhf1M6H2w/wP+uu2vXJt7LWMGjAl1OMaYHiLYPzu/D7wpIq+72+fhlgY3XUd1bTWL3l5Eemw6t51+W6jDMcb0IMEOcL8sInk4CWID8ALOjCjThfxh4x/YWbaTJRcusWVSjTEdKthCgl8BvolTKnwDMA1YS+NlVk0I7S7bzaMfPspFp13EeZnnhTocY0wPE+yYxTeBycCnqjodmAiUeBaVOSmqyv3v3E+f8D7cPcVmOBtjOl6wyaJCVSsARKSPqm4BRnkXljkZL+x8gfeK3+Nbk75FSt+UUIdjjOmBgh3gLnCvs3geeEVEDtH2sqqmExysOMjP8n/GxNSJXD7y8lCHY4zpoYId4K5brHmhiKwG4oGXPYvKBO2n637Ksepj3HfmfbZMqjHGMyd9xZaqvt52K9MZ3t77Ni/uepFbxt/C8IThoQ7HGNODefqnqIjMEZGtIrJDRJqNvIrI9SJSIiIb3NtX/B6r9dvfdO0CPP17AAAbkUlEQVTuXu9EzQnuX3s/Wf2zuGn8TaEOxxjTw3lWC0JEwoElwCygAFgnIitUdVOTps+oaqAryE6o6ulexdfdPfrhoxQcLeDJ2U/SJ7xPqMMxxvRwXp5ZTAF2qOouVa0ClgHzPHy9XmPrwa38ceMfmT9iPpPTJoc6HGNML+BlssgA9vhtF7j7mvqCiHwkIs+KyGC//dEiki8i77S03reI3Oy2yS8p6R2XfdT6alm8djHxfeL5Tt53Qh2OMaaX8DJZBFptp2nxwb8DWao6HngVeMrvsSGqmgdcA/xSRJqN4KrqY6qap6p5KSm94/qCZ7Y+w0cHPuJ7k79ny6QaYzqNl8miAPA/U8ikybUZqlqqqpXu5u+BSX6P7XV/7gLW4Fw13qsVHyvmV+//irMGncUlQy8JdTjGmF7Ey2SxDsgWkaEiEgVcBTSa1SQi6X6bc4HN7v5EEenj3k8GzgaaDoz3Oj959yf41GfLpBpjOp1ns6FUtUZEbgNWAuHAk6q6UUQWA/mqugL4hojMBWqAg8D17uG5wKMi4sNJaA8EmEXVq7z26Wus2rOKOybdweC4wW0fYIwxHUhUe8YaRnl5eZqfnx/qMDxxtOoo856fR2J0Ik9f+jSRYZGhDskY00OIyHp3fLhVtuZmN/Cr939FyYkSfjn9l5YojDEhYcWEurgPSz7kma3PcE3uNYxLGRfqcIwxvZQliy6s2lfNwrcXkto3ldsn3h7qcIwxvZh1Q3VhT218ih2Hd/DI9EeIjYwNdTjGmF7Mziy6qD1H9vC7D3/HzCEzmT5keqjDMcb0cpYsuiBVZfE7i4kMi7RlUo0xXYIliy7oxV0v8k7RO3zrjG8xMHZgqMMxxhhLFl3NoYpD/HTdT5mQMoErRl0R6nCMMQawZNHl/Cz/Z5RXldsyqcaYLsW+jbqQd4veZcXOFfz32P8mOzE71OEYY0w9SxZdREVNBYvXLmZI3BBuHn9zqMMxxphG7DqLLuKxjx7js/LP+P1Fvyc6IjrU4RhjTCN2ZtEFbD+0nT/85w/MHT6XaenTQh2OMcY0Y8kixHzqY9HaRfSL6sd3874b6nCMMSYgSxYh9n9b/48PSz7ke5O/R2J0YqjDMcaYgCxZhND+4/v55fu/ZGr6VC4ddmmowzHGmBZZsgihB957gGpfNQumLbBlUo0xXZqnyUJE5ojIVhHZISLNihyJyPUiUiIiG9zbV/weu05Etru367yMMxRWf7aaVz59hVsn3MqQ/kNCHY4xxrTKs6mzIhIOLAFmAQXAOhFZEWAt7WdU9bYmxyYB9wF5gALr3WMPeRVvZzpWfYwfv/tjRiSM4LoxPS4PGmN6IC/PLKYAO1R1l6pWAcuAeUEeOxt4RVUPugniFWCOR3F2uv/54H/Yf3w/C89aaMukGmO6BS+TRQawx2+7wN3X1BdE5CMReVZEBp/ksd3OxyUf85fNf+HKUVcyIWVCqMMxxpigeJksAo3YapPtvwNZqjoeeBV46iSORURuFpF8EckvKSk5pWA7Q7WvmkVrF5ESk8I3z/hmqMMxxpigeVnuowAY7LedCez1b6CqpX6bvwce9Dv2gibHrmn6Aqr6GPAYQF5eXrNk0tX876b/Zeuhrfzygl/SL6pfqMMxhurqagoKCqioqAh1KMZj0dHRZGZmEhnZvq5vL5PFOiBbRIYChcBVwDX+DUQkXVWL3M25wGb3/krg/xORuqvULgLu8TBWzxWUF/CbDb9h+uDpXHjahaEOxxgACgoKiIuLIysry6Zv92CqSmlpKQUFBQwdOrRdz+FZslDVGhG5DeeLPxx4UlU3ishiIF9VVwDfEJG5QA1wELjePfagiNyPk3AAFqvqQa9i9Zqq8qN3fkSYhHHv1HtDHY4x9SoqKixR9AIiwoABAziV7npPq86q6kvAS032LfC7fw8tnDGo6pPAk17G11le+uQl3tr7FvdMuYe02LRQh2NMI5YoeodT/Xe2K7g9VlZZxkPrHmJ88niuHHVlqMMxpkvp1+/Uxu4uv/xydu3a1UHRdLyqqirOO+88ampq2v0cc+bMISEhgUsvbbkkUGVlJVdeeSUjRoxg6tSp7N69u92v1xJLFh77ef7POVJ5hAVnLiA8LDzU4RjTY2zcuJHa2lqGDRsW6lBaFBUVxYUXXsgzzzzT7ue48847+fOf/9xqmyeeeILExER27NjBHXfcwV133dXu12uJJQsPrStex992/I0vj/kyo5JGhTocY7osVeXOO+9k7NixjBs3rv7L1efz8bWvfY0xY8Zw6aWXcskll/Dss88CsHTpUubNa7jO94knnmDkyJFccMEF3HTTTdx2m1MY4u9//ztTp05l4sSJzJw5k3379gGwcOFCrrvuOi666CKysrJ47rnn+N73vse4ceOYM2cO1dXVAGRlZXHvvfdy5plnkpeXx/vvv8/s2bMZPnw4v/vd7wA4evQoF154IWeccQbjxo3jhRdeqI/rsssuY+nSpe3+bC688ELi4uJabfPCCy9w3XVONYjLL7+c1157DdWOnSBqK+V5pLK2ksVrF5PZL5NbJ9wa6nCMadOiv29k094jHfqcowf1577Pj2mz3XPPPceGDRv48MMPOXDgAJMnT+a8887jrbfeYvfu3Xz88cfs37+f3NxcbrjhBgDeeustrr76agD27t3L/fffz/vvv09cXBwzZsxgwgTnotdzzjmHd955BxHh8ccf56GHHuLnP/85ADt37mT16tVs2rSJM888k+XLl/PQQw8xf/58/vGPf3DZZZcBMHjwYNauXcsdd9zB9ddfz1tvvUVFRQVjxozh1ltvJTo6mr/97W/079+fAwcOMG3aNObOnYuIMHbsWNatc+bqlJeXc+655wb8DP7yl78wevTodn3OhYWFDB7sXKkQERFBfHw8paWlJCcnt+v5ArFk4ZHHP36c3Ud28+isR4mJiAl1OMZ0aW+++SZXX3014eHhDBw4kPPPP59169bx5ptvcsUVVxAWFkZaWhrTp0+vP6aoqIiUlBQA3nvvPc4//3ySkpIAuOKKK9i2bRvgTA++8sorKSoqoqqqqtHU0YsvvpjIyEjGjRtHbW0tc+Y4VYXGjRvXqN9/7ty59fuPHj1KXFwccXFxREdHc/jwYWJjY7n33nt54403CAsLo7CwkH379pGWlkZ4eDhRUVGUl5cTFxfHhg0bOvzzC3QW0dETFyxZeGDn4Z08/vHjXDrsUs4adFaowzEmKMGcAXilpS6T1rpSYmJi6i8mbK3d7bffzre//W3mzp3LmjVrWLhwYf1jffr0ASAsLIzIyMj6L9iwsLBGg9L+7eru+7dbunQpJSUlrF+/nsjISLKyshpd6FhZWUl0dHSbZxbl5eXccsstACxevLg+SbUlMzOTPXv2kJmZSU1NDWVlZfWJs6PYmEUH86mPxWsXExsZy52T7wx1OMZ0C+eddx7PPPMMtbW1lJSU8MYbbzBlyhTOOeccli9fjs/nY9++faxZs6b+mNzcXHbs2AHAlClTeP311zl06BA1NTUsX768vl1ZWRkZGU5puaeeegovlJWVkZqaSmRkJKtXr+bTTz+tf6y0tJSUlBQiIyPrzywC3UaPHs3UqVPrt4NNFOCc+dS9t2effZYZM2bYmUVXt3z7ct7f/z73n30/SdEdm9mN6anmz5/P2rVrmTBhAiLCQw89RFpaGl/4whd47bXXGDt2LCNHjmTq1KnEx8cD8LnPfY41a9Ywc+ZMMjIyuPfee5k6dSqDBg1i9OjR9e0WLlzIFVdcQUZGBtOmTeOTTz7p8PivvfZaPv/5z5OXl8fpp59OTk5O/WOrV6/mkksuafdzn3vuuWzZsoWjR4+SmZnJE088wezZs1mwYAF5eXnMnTuXG2+8kS996UuMGDGCpKQkli1b1hFvqzFV7RG3SZMmaajtP7Zfz1x6pt7w8g3q8/lCHY4xbdq0aVOoQ2hTeXm5qqoeOHBAhw0bpkVFRaqqevz4cZ06darW1NQ0alddXa2XXnqpPvfcc6EJuIn58+frli1bQh2Gqgb+98apqNHmd6ydWXSgB9c9SGVtJT+c9kO7KtaYDnLppZdy+PBhqqqq+OEPf0hamlMFISYmhkWLFlFYWMiQIUNYuHAhr776KhUVFVx00UX1M5lCqaqqissuu4xRo7r/1HlLFh3kjYI3WLl7JbedfhtZ8VmhDseYHsN/nKKp2bNn19//2c9+1gnRnJyoqCi+/OUvhzqMDmED3B3gePVxfvTOjxgeP5wbxt4Q6nCMMabD2ZlFB/j1hl9TdKyIP138JyLDbZlUY0zPY2cWp2hj6UaWbl7KF0d+kYmpE0MdjjHGeMKSxSmo8dWw6O1FJEUn8c1JtkyqMabnsmRxCpZuXsrmg5u5Z8o99I/qH+pwjOl2rER5c5988glTp04lOzubK6+8kqqqqoDtfvKTnzBixAhGjRrFypUrOyrkFlmyaKfCo4Us2bCECzIvYNZps0IdjjG9Tk8tUX7XXXdxxx13sH37dhITE3niiSeatdm0aRPLli1j48aNvPzyy3zta1+jtra2I0NvxtNkISJzRGSriOwQkbtbaXe5iKiI5LnbWSJyQkQ2uLffeRnnyVJ3mVSAe6fea9dUGHOK1EqU138Oq1at4vLLLwfguuuu4/nnn2/W7oUXXuCqq66iT58+DB06lBEjRvDee++d1Gd+sjybDSUi4cASYBZQAKwTkRWquqlJuzjgG8C7TZ5ip6qe7lV8p2Ll7pW8Wfgmd02+i/R+6aEOx5iO8c+7ofjjjn3OtHFw8QNtNrMS5U4hwdTUVBISEoiIcL6aMzMzKSwsbNa2sLCQadOm1W+31K4jeTl1dgqwQ1V3AYjIMmAesKlJu/uBh4DvehhLhymrLOOB9x5gzIAxXJ1zdajDMaZHsBLljpKSkmb7AvVcaCeUJG/Ky2SRAezx2y4Apvo3EJGJwGBVfVFEmiaLoSLyAXAE+IGq/tvDWIP28PqHOVx5mN/O/K0tk2p6liDOALwS6Muvtf3QM0uU5+bmcvjwYWpqaoiIiKCgoIBBgwY1a1tXkrxOS+06kpdjFoHSXP2/qIiEAQ8D3wnQrggYoqoTgW8DfxGRZtONRORmEckXkfxAGbmjrd+3nuXbl/Ol0V8id0Cu569nTG9hJcqdEuUiwvTp0+vHZZ566qlG4zJ15s6dy7Jly6isrOSTTz5h+/btTJkyxZP3VsfLZFEADPbbzgT2+m3HAWOBNSKyG5gGrBCRPFWtVNVSAFVdD+wERjZ9AVV9TFXzVDWv7nTUK1W1VSxau4iMfhl8dcJXPX0tY3qb+fPnM378eCZMmMCMGTMalSjPzMxk7Nix3HLLLQFLlAONSpTPnDkzYInyc889t0OXGfV37bXXkp+fT15eHkuXLj2lEuUPPvggv/jFLxgxYgSlpaXceOONAKxYsYIFCxYAMGbMGL74xS8yevRo5syZw5IlSwgP97inI5jStO254XRx7QKGAlHAh8CYVtqvAfLc+ylAuHt/GFAIJLX2el6XKP/Nht/o2D+O1X8X/NvT1zGmM1mJcu9ZifK2k1CNiNwGrATCgSdVdaOILHaDW9HK4ecBi0WkBqgFblXVg17F2pZdZbv4/Ue/5+KhF3NOxjmhCsOYXslKlHcNoq0MDHUneXl5mp+f3+HP61MfN6y8gW2HtrHishUkx3hzGmtMKGzevJncXBt/6y0C/XuLyHpVzWvrWLuCuw3P73ie9fvW892871qiMMb0WpYsWnHgxAF+lv8zJg2cxPwR80MdjjHGhIwli1Y8tO4hKmoqWHDmAivpYYzp1SxZtODNwjf55yf/5KZxNzEsvusWKjPGmM5gySKAumVSh8YP5cZxN4Y6HGN6LCtR3tyvf/1rRowYgYhw4MCBFts99dRTZGdnk52d7dnFhv4sWQTw2w9/S+HRQu478z6iwqNCHY4xJoCeWqL87LPP5tVXX+W0005rsc3BgwdZtGgR7777Lu+99x6LFi3i0KFDHRFyiyxZNLG5dDN/3vRnvpD9BSYNnBTqcIzpFdRKlNebOHEiWVlZrbZZuXIls2bNIikpicTERGbNmsXLL78c9Gu0h5eFBLudWl8ti9YuIqFPAndMuiPU4RjTqR5870G2HNzSoc+Zk5TDXVPuarOdlSh3CgmOHj06qM+1sLCQwYMbqil19xLl3c7TW55mY+lGfnreT4nvEx/qcIzpNaxE+ckJdDF1dy5R3q0UHS3ikQ8e4dyMc5mdNTvU4RjT6YI5A/BKS5UkWqsw0RNLlAd7ZpGZmdmoAm9BQQEXXHBBUMe2l41Z4PxH+/G7Pwbg+9O+b9dUGNPJrET5hqATBcDs2bP517/+xaFDhzh06BD/+te/mD3b2z9yLVkAr3z6Cq8XvM7XT/86Gf0yQh2OMb2OlShv8Mgjj5CZmUlBQQHjx4/nK1/5CgD5+fn195OSkvjhD3/I5MmTmTx5MgsWLKjvgvNMMKVpu8OtvSXKyyrLdPoz0/WKFVdodW11u57DmO7KSpR7z0qU9xBVtVWMTR7LLRNuISKs138cxnQ5VqK8a7AS5cb0YlaivHexEuXGGGM8ZcnCmF6up/QumNad6r+zp8lCROaIyFYR2SEid7fS7nIRURHJ89t3j3vcVhGxCx+M8UB0dDSlpaWWMHo4VaW0tJTo6Oh2P4dnI7oiEg4sAWYBBcA6EVmhqpuatIsDvgG867dvNHAVMAYYBLwqIiNVtdareI3pjeqmaJaUlIQ6FOOx6OhoMjMz2328l9N/pgA7VHUXgIgsA+YBm5q0ux94CPiu3755wDJVrQQ+EZEd7vOt9TBeY3qdyMjIRuUvjGmJl91QGcAev+0Cd189EZkIDFbVF0/2WGOMMZ3Hy2QRqGZGfceoiIQBDwPfOdlj/Z7jZhHJF5F8O402xhjveJksCoDBftuZwF6/7ThgLLBGRHYD04AV7iB3W8cCoKqPqWqequbVVZ80xhjT8Ty7KE9EIoBtwIVAIbAOuEZVN7bQfg3wXVXNF5ExwF9wxikGAa8B2a0NcItICfBpS48HIRloeQ3D0LG4To7FdXIsrpPTE+M6TVXb/GvbswFuVa0RkduAlUA48KSqbhSRxTi1SFa0cuxGEfkrzmB4DfD1tmZCBfNmWyMi+cFcxdjZLK6TY3GdHIvr5PTmuDwthqSqLwEvNdm3oIW2FzTZ/jHwY8+CM8YYEzS7gtsYY0ybLFk0eCzUAbTA4jo5FtfJsbhOTq+Nq8dUnTXGGOMdO7MwxhjTpl6bLETkChHZKCI+/wKGAdoFVQyxA+NKEpFXRGS7+zOxhXa1IrLBvbU4s6wD4mn1/YtIHxF5xn38XRHJ8iqWk4jpehEp8ft8vuJ1TO7rPiki+0XkPy08LiLyiBv3RyJyRheJ6wIRKfP7vAJOQvEgrsEislpENru/i98M0KbTP7Mg4+r0z0xEokXkPRH50I1rUYA23v0+BrOcXk+8AbnAKGANkNdCm3BgJzAMiAI+BEZ7HNdDwN3u/buBB1tod7QTPqM23z/wNeB37v2rgGe6QEzXA78Owf+p84AzgP+08PglwD9xKhRMA97tInFdALwYgs8rHTjDvR+Hc11W03/LTv/Mgoyr0z8z9zPo596PxCm+Oq1JG89+H3vtmYWqblbVrW00qy+GqKpVQF0xRC/NA55y7z8FhHJtyGDev3+8zwIXikigci2dGVNIqOobwMFWmswD/qSOd4AEEUnvAnGFhKoWqer77v1yYDPNa8B1+mcWZFydzv0Mjrqbke6t6aCzZ7+PvTZZBCkUBQ0HqmoROP9pgdQW2kW7dbHeERGvEkow77++jarWAGXAAI/iCTYmgC+43RbPisjgAI+HQlcukHmm273xT7eCQqdyu0sm4rdUgSukn1krcUEIPjMRCReRDcB+4BVVbfHz6ujfR08vygs1EXkVSAvw0PdV9YVgniLAvlOePtZaXCfxNENUda+IDANWicjHqrrzVGNrIpj378ln1IpgXu/vwNOqWikit+L8pTXDw5iC1dmfVbDexyn5cFRELgGeB7I768VFpB+wHPiWqh5p+nCAQzrlM2sjrpB8ZupUsjhdRBKAv4nIWFX1H4vy7PPq0clCVWee4lMEVdDwZLUWl4jsE5F0VS1yT7f3t/Ace92fu8SpqzURpy+/IwXz/uvaFIhTDyweb7s82oxJVUv9Nn8PPOhhPCfDk/9Pp8r/i1BVXxKR34hIsqp6XgNJRCJxvpCXqupzAZqE5DNrK65Qfmbuax52f+/nAP7JwrPfR+uGat06IFtEhopIFM6AkWczj1wrgOvc+9cBzc6ARCRRRPq495OBs2m+qFRHCOb9+8d7ObBK3dE1j7QZU5M+7bk4fc5dwQrgy+4Mn2lAWV2XYyiJSFpdv7aITMH5Xiht/agOeV0BngA2q+ovWmjW6Z9ZMHGF4jMTkRT3jAIRiQFmAluaNPPu97EzR/O70g2Yj5OFK4F9wEp3/yDgJb92l+DMhtiJ033ldVwDcKrsbnd/Jrn784DH3ftnAR/jzAT6GLjRw3iavX9gMTDXvR8N/B+wA3gPGNYJn1FbMf0E2Oh+PquBnE76P/U0UARUu/+3bgRuBW51HxecpYZ3uv9uAWfhhSCu2/w+r3eAszoprnNwukg+Aja4t0tC/ZkFGVenf2bAeOADN67/AAvc/Z3y+2hXcBtjjGmTdUMZY4xpkyULY4wxbbJkYYwxpk2WLIwxxrTJkoUxxpg2WbIw5iSIyNG2W7V6/LPuVfeISD8ReVREdrpVRN8QkakiEuXe79EXzZruxZKFMZ3ErR8Urqq73F2P41xdm62qY3Cq5SarUyDxNeDKkARqTACWLIxpB/eK4p+KyH9E5GMRudLdH+aWftgoIi+KyEsicrl72LW4V+SLyHBgKvADVfWBU7pFVf/htn3ebW9Ml2Cnuca0z/8DTgcmAMnAOhF5A6f0ShYwDqdi8GbgSfeYs3GupgYYA2xQpzBcIP8BJnsSuTHtYGcWxrTPOTiVbWtVdR/wOs6X+znA/6mqT1WLccqN1EkHSoJ5cjeJVIlIXAfHbUy7WLIwpn1aWlCmtYVmTuDU7gGnrtAEEWntd7APUNGO2IzpcJYsjGmfN4Ar3cVoUnCWLn0PeBNn4aUwERmIs/xmnc3ACAB11h7JBxb5VS/NFpF57v0BQImqVnfWGzKmNZYsjGmfv+FU//wQWAV8z+12Wo5T2fU/wKM4K6yVucf8g8bJ4ys4i2DtEJGPcdbeqFurYTrwkrdvwZjgWdVZYzqYiPRTZwW1AThnG2erarG7BsFqd7ulge2653gOuEfbXifemE5hs6GM6XgvuovURAH3u2ccqOoJEbkPZ53kz1o62F3U6XlLFKYrsTMLY4wxbbIxC2OMMW2yZGGMMaZNliyMMca0yZKFMcaYNlmyMMYY0yZLFsYYY9r0/wMeg33ISVMlnQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#未降维\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from matplotlib import pyplot as plt\n",
    "train=pd.read_csv(\"Otto_FE_train_tfidf.csv\")\n",
    "#按照题目采用train_test_split抽取数据\n",
    "from sklearn.model_selection import train_test_split\n",
    "train,train_drop=train_test_split(train,train_size=10000,random_state=0)\n",
    "\n",
    "y_train=train['target']\n",
    "X_train=train.drop([\"id\",\"target\"],axis=1)\n",
    "\n",
    "train_id=train['id']\n",
    "\n",
    "feat_names=X_train.columns\n",
    "#数据用稀疏矩阵表示，模型训练变快\n",
    "from scipy.sparse import csr_matrix\n",
    "X_train=csr_matrix(X_train)\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train_part,X_val,y_train_part,y_val=train_test_split(X_train,y_train,train_size=0.8,random_state=0)\n",
    "\n",
    "\n",
    "#数据用稀疏矩阵表示，模型训练变快\n",
    "from scipy.sparse import csr_matrix\n",
    "X_train=csr_matrix(X_train)\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train_part,X_val,y_train_part,y_val=train_test_split(X_train,y_train,train_size=0.8,random_state=0)\n",
    "\n",
    "#超参数调优，正则参数C,RBF核函数的宽度\n",
    "from sklearn.svm import SVC\n",
    "def fit_grid_point_RBF(C,gamma,X_train,y_train,X_val,y_val):\n",
    "    SVC3=SVC(C=C,kernel='rbf',gamma=gamma)\n",
    "    SVC3=SVC3.fit(X_train,y_train)\n",
    "    accuracy_train=SVC3.score(X_train,y_train)\n",
    "    accuracy=SVC3.score(X_val,y_val)\n",
    "    print(\"C={} and gamma={}:,accuracy_train={}，accuracy_val={}\" .format(C,gamma,accuracy_train,accuracy))\n",
    "    return accuracy\n",
    "\n",
    "accuracy_s=np.matrix(np.zeros(shape=(5,3)),float)\n",
    "\n",
    "#需要调整的参数\n",
    "gamma_s=np.logspace(-1,1,3)\n",
    "\n",
    "C_s=np.logspace(-1,3,5)#吧10的-1次方到10的3次方分成5份\n",
    "\n",
    "\n",
    "for i, oneC in enumerate(C_s):\n",
    "    for j,gamma in enumerate(gamma_s):\n",
    "        tmp=fit_grid_point_RBF(oneC,gamma,X_train_part,y_train_part,X_val,y_val)\n",
    "        accuracy_s[i,j]=tmp\n",
    "        \n",
    "#数据可视化        \n",
    "x_axis=np.log10(C_s)\n",
    "for j,gamma in enumerate(gamma_s):\n",
    "    plt.plot(x_axis,np.array(accuracy_s[:,j]),label='log(gamma)='+str(np.log10(gamma)))\n",
    "plt.legend()\n",
    "plt.xlabel('log(C)')\n",
    "plt.ylabel('accuracy')\n",
    "plt.savefig('RBF_SVM_PCA_Otto.png')\n",
    "\n",
    "plt.show() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最优结果：C=10.0 and gamma=1.0:,accuracy_train=0.914，accuracy_val=0.7875\n",
    "未降维最优结果：C=10.0 and gamma=1.0:,accuracy_train=0.950625，accuracy_val=0.7885  \n",
    "\n",
    "降维后运行速度较快，结果稍差一点"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
